Skip to content

Find files to check in Rust - #2709

Open
somewhatabstract wants to merge 13 commits into
claude/jolly-galileo-72zesp-glob-braces-dotfrom
claude/jolly-galileo-72zesp-get-files
Open

somewhatabstract wants to merge 13 commits into
claude/jolly-galileo-72zesp-glob-braces-dotfrom
claude/jolly-galileo-72zesp-get-files

Conversation

@somewhatabstract

@somewhatabstract somewhatabstract commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Stacked on #2708. This adds get_files, the Rust port of checksync's getFiles, and finishes Phase 2 of #2615.

New file_search module, written by me:

  • Planning: plan_glob_tasks does the following:

    • expands brace groups;
    • treats ! includes and ! excludes as excludes;
    • collapses repeated slashes and drops leading ./;
    • works out the folder each include walks from.

    expand_folder_pattern turns a folder named without a * into folder/**, as the TypeScript does.

  • Walking: walk_glob_folder walks each starting folder once, for all the patterns that start there.

    • It lists regular files only, and doesn't follow links it finds on the way. A pattern's starting folder can be a link.
    • It enters hidden folders only for a pattern folder name that could match one, stops at the depth a pattern without ** allows, and prunes excluded folders.
    • Read errors name the path that failed.
  • Results: find_files sorts results as JavaScript sorts strings. get_files loads the ignore files first, then filters with them.

  • Matching rules: a pattern matches a path that is exactly its text as well as by glob, so app/[slug]/page.tsx works. Includes match the way fast-glob does by default:

    • by folder;
    • case-sensitively;
    • by whole characters;
    • with wildcards matching no name that starts with ., and never running over . or ...

    Excludes also match names that start with ., as in fast-glob. Absolute excludes are matched against each path's absolute form, as in fast-glob.

glob module, from the clean-room agent:

  • Add brace expansion and a dotfile option to the Rust glob module #2708 review points: two commits fold them in. A NUL ends a brace pattern, and the test-table test reports instead of crashing. They also expose brace expansion.
  • unicode option: makes ? and bracket sets match one whole character for file search. gitignore stays byte-based, as git is.
  • HiddenNames option: replaces the old dot flag. It keeps * and ** off . and .. names for fast-glob matching, so **/one.js no longer matches ../src/one.js. gitignore keeps git's rule. The fastglob-vectors.json overrides for ./.. are now agreements. This also removes the struct_excessive_bools exception.

Deliberate differences, as decided for 12.0.0:

  • An exclude always prunes the folders it matches. fast-glob only does so for excludes without wildcards, and never for a pattern's starting folder.
  • x/** doesn't match x itself.
  • ? is a wildcard in any folder name. fast-glob treats it literally in a folder name before the last one.
  • A negated bracket set doesn't match the . that starts a hidden name. micromatch's does.
  • A pattern that walks into a file finds nothing. The TypeScript crashes with "not a directory".
  • An emoji counts as one character. micromatch counts it as two.

The first four are recorded in file-vectors.json with rust reasons. There are also new vectors for:

  • literal paths;
  • ! excludes;
  • repeated slashes;
  • bracket sets and escaped dots on hidden folders;
  • .. with wildcard and step-up excludes;
  • ?.js against é.js.

Test plan

  • The new Rust test tests/file_vectors.rs passes all 46 file vectors. The TypeScript test passes with them too.
  • I compared get_files with fast-glob on 3,000 random file trees and patterns. Every difference comes from the deliberate rules above: 44 from the folder rule and 26 from a trailing /**.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn

expand_braces now treats a NUL byte as the end of the pattern, as
compile_glob does, so nothing after it is expanded or counted against
the limits. The fast-glob vector test compares expected errors and
reports a mismatch instead of panicking on one.

expand_braces and BraceExpansionError are re-exported from the glob
module for the file finder, with narrow expectations until it calls
them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn
Add a file_search module with a public get_files, the Rust port of
checksync's getFiles:

- plan_glob_tasks expands brace groups, turns `!` includes into excludes,
  drops leading `./`, and finds the folder each include pattern walks from.
- expand_folder_pattern turns a folder named without a `*` into
  `folder/**`, as the JavaScript does.
- walk_glob_task walks a task's folder without following links found on
  the way, lists only regular files, skips hidden folders a pattern can't
  match, stops at the depth a pattern without `**` allows, and prunes
  excluded folders.
- find_files runs every task and sorts the results as JavaScript sorts
  strings; load_ignore_files and get_files add ignore-file filtering.

Includes match with fast-glob's defaults: by folder, case-sensitively, and
with wildcards not matching names starting with `.`. Excludes also match
those names, as in fast-glob. Unlike fast-glob, an exclude always prunes
the folders it matches, and `x/**` doesn't match `x` itself.

This also drops the temporary lint expectations on expand_braces and its
re-exports, now that get_files calls it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn
tests/file_vectors.rs builds the vectors' tree, runs get_files for each
vector and compares with the TypeScript's results, or the `rust` result
where the vector records a deliberate difference.

Three new vectors record those differences: an exclude that names a
pattern's starting folder, a wildcard exclude that matches a folder, and a
trailing `/**` against a file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn
GlobOptions gains `unicode`. With it on, as in fast-glob, `?` and
bracket sets match one whole UTF-8 character, sets compare code points,
`*` never stops inside a character, and a byte that isn't valid UTF-8
counts as one character. With it off, matching is unchanged; gitignore
turns it off. The single-unit matcher is renamed to match, and bracket
members now hold codes rather than bytes.

fastglob-vectors.json gains multi-byte cases from micromatch, and
records that micromatch counts the two UTF-16 halves of an emoji
separately.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn
File search sets the glob matcher's new unicode option, so `?` and
bracket sets match one whole character, as fast-glob's do, rather than one
byte. A new shared vector checks that `?.js` matches `é.js`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn
@changeset-bot

changeset-bot Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 969a99a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Windows doesn't allow `*` in a folder name, so the test that a pattern with a star is never treated as a folder can't create its folder there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn
@codecov

codecov Bot commented Nov 12, 2026

Copy link
Copy Markdown

Bundle Report

Bundle size has no change ✅

The vectors write link targets with `/`, which Windows can't follow, so get_files failed to read through `links/dir`. The fixture now converts them, as Node's symlink does for the TypeScript test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn
@codecov

codecov Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (3f070b3) to head (969a99a).

Additional details and impacted files

Impacted file tree graph

@@                               Coverage Diff                               @@
##           claude/jolly-galileo-72zesp-glob-braces-dot     #2709     +/-   ##
===============================================================================
  Coverage                                       100.00%   100.00%             
===============================================================================
  Files                                              126       154     +28     
  Lines                                             5085      6877   +1792     
  Branches                                          4293      6083   +1790     
===============================================================================
+ Hits                                              5085      6877   +1792     
Flag Coverage Δ
rust 100.00% <100.00%> (ø)
typescript 100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...ecksync-core/src/file_search/absolute_glob_text.rs 100.00% <100.00%> (ø)
...sync-core/src/file_search/bracket_may_match_dot.rs 100.00% <100.00%> (ø)
...checksync-core/src/file_search/collapse_slashes.rs 100.00% <100.00%> (ø)
...sync-core/src/file_search/compile_exclude_globs.rs 100.00% <100.00%> (ø)
...cksync-core/src/file_search/compile_pattern_set.rs 100.00% <100.00%> (ø)
...sync-core/src/file_search/expand_folder_pattern.rs 100.00% <100.00%> (ø)
...sync-core/src/file_search/expand_pattern_braces.rs 100.00% <100.00%> (ø)
...rates/checksync-core/src/file_search/find_files.rs 100.00% <100.00%> (ø)
...crates/checksync-core/src/file_search/get_files.rs 100.00% <100.00%> (ø)
...crates/checksync-core/src/file_search/glob_base.rs 100.00% <100.00%> (ø)
... and 33 more

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3f070b3...969a99a. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…der once

- Match a pattern's exact text as well as its glob, so paths like
  `app/[slug]/page.tsx` are found and excluded.
- Keep a relative form of absolute excludes, and treat a `!` exclude
  as an exclude.
- Collapse repeated slashes in patterns.
- Report read errors with the path that failed.
- Load ignore files before planning, and walk each base folder once.
- Let a non-negated bracket set at the start of a name enter hidden
  folders.
- Add shared vectors for these cases, and record `?` in a folder name
  as a deliberate difference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn
Replace GlobOptions' `dot` with `hidden`, a HiddenNames choice of which
hidden names wildcards may match: every name, as in git; names starting
with `.` but not `.` or `..`, as in fast-glob with `dot: true`; or none,
as in fast-glob's default. In the last two, a `*` or `**` can't start at
or run over any part of a name that is exactly `.` or `..`, while a
literal `..` still matches it. gitignore keeps git's rule.

fastglob-vectors.json now runs `dot: true` cases with the new rule, so
most `.`/`..` divergences become agreements; one remaining difference,
`.*` against `.`, is recorded, along with new `..` cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn
Use `SkipDotNames` for includes and `SkipDotSegments` for excludes, so
a wildcard exclude like `**/one.js` no longer matches `../src/one.js`,
as in fast-glob. Add a shared vector for it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn
- Match absolute exclude patterns against each path's absolute form, as
  fast-glob does, so wildcards above the working folder work, like
  `/repo/**/gen/**` from `/repo/src`. This replaces rewriting them
  relative to the working folder.
- Walk hidden folders only for a folder name that may match one: a
  literal `.`, or a bracket set listing a `.` or with a range covering
  one. File names and sets like `[A-Z]` no longer make every walk
  enter `.git`.
- Collapse three or more leading slashes to two.
- Compile each exclude once, keeping only the forms with `/**` stripped
  for folders.
- Bring a glob comment up to date with the `.`/`..` rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn
… dots

- A `..` or `.` on the way to a walk's folder doesn't name a folder its
  files are in, so excludes no longer prune the walk there. An include
  like `../../other/*.js` with an exclude matching `..` finds files
  again, as in fast-glob.
- A folder name starting with an escaped `.`, like `\.c*`, can match a
  hidden folder, so walking enters them for it.
- Test absolute paths' text on Windows.
- Add shared vectors for both, and record that a negated bracket set
  can't match a leading dot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QjLhVH1NnUc7MzCSYLd8Yn
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants