Skip to content

feat: add issues module with an early implementation for the filename validator - #86

Open
karellopez wants to merge 1 commit into
bids-standard:mainfrom
karellopez:main
Open

feat: add issues module with an early implementation for the filename validator#86
karellopez wants to merge 1 commit into
bids-standard:mainfrom
karellopez:main

Conversation

@karellopez

Copy link
Copy Markdown
Collaborator

Closes #85

What this adds

Two modules that turn filename validation into structured findings.

Module Role
bids_validator/issues.py The container: what a finding is. Pure data, no I/O.
bids_validator/filename_checks.py The logic: schema-driven filename and path checks.

Validation currently answers a yes/no question. BIDSValidator.is_bids() returns a
boolean and the CLI prints one line per bad file, which carries no severity, no stable
code, and no structure a program can consume. context.py already anticipates this:
ValidationError is a stub whose body is """TODO: Add issue structure.""".

A problem is now a typed record:

[error] DATATYPE_MISMATCH
    file   : sub-01/func/sub-01_T1w.nii.gz
    detail : the file is in 'func' but its suffix belongs in: anat
    rule   : rules.files.raw.anat.nonparametric

Scope is names and paths only. Nothing here opens a file or reads its contents.
BIDSValidator.is_bids() is untouched, since pybids and mne-bids depend on it.

This PR adds 6 files and modifies none.

Design notes

  • attrs, not pydantic or msgspec, matching context.py and types/files.py.
  • The two modules are separate so issues.py imports nothing from the package and can
    never take part in an import cycle. Any future check module can depend on it.
  • filename_checks.py reads rules.files from the schema and identifies which rule(s) a
    path matches, then reports one specific code per kind of failure. Nothing about BIDS is
    hardcoded; only the ten issue code strings are constants, because the schema does not
    define them. They come from the TypeScript validator's catalog (src/issues/list.ts),
    mirrored in FILENAME_ISSUES so the provenance is explicit and the output stays
    interchangeable.
  • The walk applies the TypeScript validator's default ignores (.git**, .*,
    sourcedata/, code/, stimuli/, log/) in addition to .bidsignore. Without them,
    dotfiles such as .DS_Store are reported, which the TypeScript validator never does.
  • Directory recordings (CTF .ds, MEF .mefd, OME-Zarr) are treated as single units: the
    walk does not descend into them.
  • Schema lookups are memoised per schema object, so the rule tree is flattened once rather
    than once per file (measured: 0.8 ms first call, 0.0005 ms thereafter, 179 rules). The
    walk is a generator, so a large dataset holds one context at a time.

Field naming and JSON output

Following your note in #85: the model uses snake_case internally, and reporting output
should follow the TypeScript validator's camelCase interface. Four of the six fields
already match; the mapping for a future reporter is:

this model TypeScript validator
code code
severity severity
location location
rule rule
sub_code subCode
message issueMessage

No JSON reporter is included in this PR, so nothing emits either form yet.
attrs.asdict(issue) produces the internal names and is only a convenience for
inspection, not a reporting format. I will add the camelCase mapping when the reporter
lands, or I can add a small to_reporting_dict() here if you would rather the convention
were in place from the start.

One deliberate difference from the TypeScript validator

A data file outside any recognised datatype directory is reported as INVALID_LOCATION:

sub-01/foo/sub-01_T1w.nii.gz     "foo" is not a datatype
sub-01/sub-01_T1w.nii.gz         no datatype folder at all

The TypeScript validator misses this. findDatatype returns an empty string when the
parent folder is not a known datatype (src/schema/datatypes.ts), and DATATYPE_MISMATCH
is gated on that value being truthy (src/validators/filenameValidate.ts). Meanwhile
suffix-based rule matching ignores the datatype entirely
(src/validators/filenameIdentify.ts), so the file still matches a rule and is not
NOT_INCLUDED either.

The legacy is_bids regexes do catch it, because they cover the whole path. Since
this module is the schema-driven successor to that check, dropping the case would lose
coverage users already have:

Path is_bids TypeScript validator this PR
/sub-01/anat/sub-01_T1w.nii.gz True clean clean
/sub-01/foo/sub-01_T1w.nii.gz False clean INVALID_LOCATION
/sub-01/sub-01_T1w.nii.gz False clean INVALID_LOCATION
/sub-01/func/sub-01_T1w.nii.gz False DATATYPE_MISMATCH DATATYPE_MISMATCH

Metadata files are exempt: the inheritance principle lets a .json or .tsv sit higher
in the tree than the data it describes, so sub-01/sub-01_T1w.json and
task-rest_bold.json at the dataset root are correct and are not flagged.

Happy to put this behind a flag if you would rather keep strict parity with the
TypeScript validator by default. I think it is also worth reporting upstream to
bids-standard/bids-validator as a genuine miss.

Tests

Following the existing conventions: plain pytest, the session-scoped schema fixture
from tests/conftest.py, @pytest.mark.parametrize, no new fixture machinery and no new
dependency. No datalad usage added (per #10).

  • tests/test_issues.py covers the model: defaults, both severities, the collection
    helpers, and an attrs.asdict round trip.
  • tests/test_filename_checks.py is table driven with one case per issue code, plus
    the default ignores, .bidsignore, the rule field, catalog completeness, and the
    datatype-directory cases cross-checked against is_bids to pin the intent.

28 tests, all passing.

CI status

build-test-deploy passes on the full Python 3.10 to 3.14 matrix:

The style job fails, for reasons that predate this PR. ruff check src/ reports 8
findings, none of them in the files this PR adds:

File Findings
src/bids_validator/context.py 6
src/bids_validator/bids_validator.py 1
src/bids_validator/types/_typings.py 1
src/bids_validator/issues.py (added here) 0
src/bids_validator/filename_checks.py (added here) 0

Rules: PIE790 x2, PLW0120, PYI036, RUF022, SIM102, SIM118, TRY004.

I verified this is not caused by the PR by checking out main at 5a361c4 in a clean
worktree, where neither added module exists, and running the same command. It produces
the identical Found 8 errors with the same rules in the same three files. The cause is
that tox.ini declares deps = ruff unpinned, so CI installs the newest ruff (0.16.2 in
this run) and recently added rules flag existing code, the same situation as the earlier
chore: Resolve ruff complaints commit.

For the two modules and two test files added here, ruff check, ruff format --diff,
codespell and mypy are all clean.

I have deliberately not fixed those 8 findings in this PR, to keep the diff to the
feature. Happy to send them as a separate chore PR if useful. One caveat if I do:
TRY004 asks to change a ValueError to a TypeError in context.py, which is a
behaviour change, so I would rather you decide that one than have me slip it into a
cleanup.

Coverage

Measured locally on the added modules with the same invocation CI uses:

Name                                    Stmts   Miss Branch BrPart  Cover
-------------------------------------------------------------------------
src/bids_validator/filename_checks.py     279     14    130     18    92%
src/bids_validator/issues.py               32      0      0      0   100%
-------------------------------------------------------------------------
TOTAL                                     311     14    130     18    93%

93% on the added code, above the 80% project and patch targets in codecov.yml.

Docs

  • docs/filename_issues_module.md: what the modules add, architecture with flowcharts,
    the ten codes, usage, how content validation extends the same layer, and a technical
    reference covering every type and function with the reasoning behind each decision.
  • docs/example_filename_issues.py: runs two ways. With no argument it generates a
    dataset containing one deliberately broken file per issue code; with a path it
    validates your own dataset.

Not included, on purpose

Validation currently answers a yes/no question: BIDSValidator.is_bids returns a
boolean and the CLI prints one line per bad file. That output carries no severity,
no stable code, and no structure a program can consume. These two modules replace
it with typed findings, and give the future content checks somewhere to report to.

Engine

- issues.py holds the finding model: Severity, Issue, and a DatasetIssues
  collection. Pure attrs data with no I/O, matching the convention already used by
  context.py and types/files.py. The field set mirrors the reference validator so
  output stays interchangeable, and Issue.rule records which schema rule fired.
- filename_checks.py holds the check logic, kept separate from the container so
  each has one job. It reads rules.files from the schema, identifies the rule or
  rules a path matches, then reports one specific code per kind of failure rather
  than a single blanket "bad name": NOT_INCLUDED, MISSING_REQUIRED_ENTITY,
  ENTITY_NOT_IN_RULE, ENTITY_WITH_NO_LABEL, INVALID_ENTITY_LABEL,
  EXTENSION_MISMATCH, DATATYPE_MISMATCH, INVALID_LOCATION, FILENAME_MISMATCH and
  ALL_FILENAME_RULES_HAVE_ISSUES.
- Scope is names and paths only. Nothing opens a file or reads its contents.
- The walk applies the reference validator's default ignores (.git**, .*,
  sourcedata/, code/, stimuli/, log/) in addition to .bidsignore. Without them
  dotfiles such as .DS_Store are reported, which the reference never does.
- Directory recordings such as CTF .ds are treated as single units: the walk does
  not descend into them and does not name-check their contents.
- Schema lookups are memoised per schema object, so the rule tree is flattened
  once rather than once per file, and the walk is a generator so a large dataset
  holds one context at a time.

One deliberate difference from the reference validator

- A data file outside any recognised datatype directory is reported as
  INVALID_LOCATION. The reference misses this case, because its suffix matching
  ignores the datatype and its DATATYPE_MISMATCH check is gated on the parent
  directory being a known datatype. The legacy is_bids regexes do catch it, since
  they cover the whole path, so dropping it would lose coverage users already
  have. Metadata files are exempt, because the inheritance principle lets a .json
  or .tsv sit higher in the tree than the data it describes.

Tests

- test_issues.py covers the model: defaults, severities, the collection helpers
  and a serialisation round trip.
- test_filename_checks.py is table driven with one case per issue code, plus the
  default ignores, .bidsignore, the rule field, catalog completeness, and the
  datatype-directory case cross-checked against is_bids.
- 28 tests pass. ruff, ruff format and mypy strict are clean.
- Verified on real MRI, EEG, MEG and PET datasets: the filename findings match
  those of a reference-parity engine exactly.

Docs

- docs/filename_issues_module.md explains what the modules add, the architecture
  with flowcharts, the ten codes, how to run it, how content validation extends
  the same layer, and a technical reference covering every type and function with
  the reasoning behind each decision.
- docs/example_filename_issues.py runs two ways: with no argument it generates a
  dataset containing one deliberately broken file per issue code, and with a path
  it validates your own dataset.
@codecov

codecov Bot commented Aug 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.45283% with 32 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.29%. Comparing base (5a361c4) to head (6955a54).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #86      +/-   ##
==========================================
+ Coverage   90.56%   91.29%   +0.73%     
==========================================
  Files          13       17       +4     
  Lines         890     1310     +420     
  Branches      130      219      +89     
==========================================
+ Hits          806     1196     +390     
- Misses         50       62      +12     
- Partials       34       52      +18     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a structured issues model and emit findings from the filename validator

1 participant