Skip to content

feat(search): say what never made it into the index (#816) - #856

Draft
hr1juldey wants to merge 1 commit into
trailhq:mainfrom
hr1juldey:fix/816-parse-failure-honesty
Draft

hr1juldey wants to merge 1 commit into
trailhq:mainfrom
hr1juldey:fix/816-parse-failure-honesty

Conversation

@hr1juldey

Copy link
Copy Markdown

Fixes #816.

What happens today

graft grep with parse failures in the graph answers as if the index were complete: no hits for "class Foo" in 3897 indexed files. The pattern may be too specific…. A miss is blamed on the pattern while 2,321 files were never searched — an agent reads that as "no other usages exist".

Fix

  • build: meta.parseFailed: {path, error}[] rides the graph — fresh throws AND cached failures re-parsed on the run (Transient grammar aborts are cached as per-file parse errors and replayed forever: a repo silently loses a whole language until --no-reuse #312 re-parses, never replays, an errored entry). A clean build writes nothing → old graphs behave identically.
  • grep / find_all (CLI + MCP share the formatters): header and zero-hit note append N source files failed to parse and were not searched (run graft build --no-reuse, or use rg for exhaustive search); the zero-hit note REPLACES "All indexed code was searched", which is a lie while failures ride the graph. Federated grep sums children counts. --json carries parseFailed.
  • skeleton: a failed file gets this file failed to parse at build time, so nothing was indexed — <the build error> instead of "no definitions indexed for this file".

Tests

Build-side uses the #139 throwing-grammar seam (fresh build, re-parse, clean build absence); grep-side asserts the sentences appear/disappear and that an old graph without meta.parseFailed answers byte-identical to today; skeleton-side covers exact path, basename suffix, and a parsed file being untouched.

When a build records parse failures, grep/find_all/skeleton answered as
if the index were complete — a miss was blamed on the pattern and the
header counted only indexed files, so an agent read 'no hits' as 'no
other usages exist' while a third of the repo never made it into the
graph (the C# abort storm of trailhq#569/trailhq#312 is the reported case).

- buildGraph now carries the failed files on meta.parseFailed (path +
  error), fresh throws and cached failures re-parsed on the run alike;
  a clean build writes nothing, so old-graph behavior is unchanged.
- grep/find_all headers and zero-hit notes append 'N source files
  failed to parse and were not searched (run graft build --no-reuse,
  or use rg for exhaustive search)'; the zero-hit note replaces 'All
  indexed code was searched', which is a lie while failures ride the
  graph. Federated grep sums the children's counts.
- skeleton names the build error for a failed file instead of 'no
  definitions indexed for this file'.

The count rides the result (--json included) so a caller can tell an
exhaustive-searach was impossible; trailhq#370's oversize skips can attach to
the same field when that lands.
@github-actions

Copy link
Copy Markdown
Contributor

🌱 graft blast radius

3 areas changed → 4 areas can be affected. 15 dependent symbols, depth 2.
Tests: 3 areas updated their tests.
Tag: @anirudhkumar-nanonets — 7 of 7 areas · @shhdwi — 7 of 7 areas

flowchart TB
  A0(("Graph Refresh<br/>6 symbols"))
  A1(("Pull Request Review<br/>4 symbols"))
  A2(("MCP Tool Dispatch<br/>3 symbols"))
  A3(("CLI Entry Point<br/>2 symbols"))
  classDef reached fill:#D9EDF3,stroke:#3AA7C9,stroke-width:1.5px,color:#0E313C;
  class A0,A1,A2,A3 reached;
Loading
Can be affected Symbols Nearest hop Reached from
Graph Refresh 6 src/graph/refresh-run.ts:L18-L21 main — calls, depth 1 Workspace Graph, Grep Search
Pull Request Review 4 src/app/review.ts:L45-L99 reviewPullRequest — calls, depth 1 Workspace Graph
MCP Tool Dispatch 3 src/mcp/tools.ts:L265-L351 callSingleTool — calls, depth 1 skeleton, Workspace Graph +1
CLI Entry Point 2 src/cli.ts:L1-L3499 cli.ts — calls, depth 1 skeleton, Workspace Graph +1
Who knows this code — 2 people across 7 areas
Area Who knows it
Workspace Graph · changed @anirudhkumar-nanonets — 22 commits, last 3mo ago · @shhdwi — 14 commits, last 22d ago
Grep Search · changed @shhdwi — 5 commits, last 27d ago · @anirudhkumar-nanonets — 5 commits, last 28d ago
skeleton · changed @anirudhkumar-nanonets — 17 commits, last 16d ago · @shhdwi — 18 commits, last 27d ago
Graph Refresh · affected @anirudhkumar-nanonets — 11 commits, last 16d ago · @shhdwi — 6 commits, last 22d ago
Pull Request Review · affected @anirudhkumar-nanonets — 11 commits, last 1mo ago · @shhdwi — 2 commits, last 22d ago
MCP Tool Dispatch · affected @shhdwi — 17 commits, last 27d ago · @anirudhkumar-nanonets — 8 commits, last 28d ago
CLI Entry Point · affected @anirudhkumar-nanonets — 77 commits, last yesterday · @shhdwi — 52 commits, last 22d ago

Ownership is git history over each area's own files, weighted towards recent work (120-day half-life). Merge commits and bots are dropped, and you are dropped from your own PR. A name with no @ has no GitHub handle in its commit email — tag them by hand, or add a .mailmap entry. A suggestion from history, not a CODEOWNERS rule.

All 15 dependent symbols, grouped by area

Graph Refresh — 6 symbols in 3 files

  • src/graph/refresh-run.ts:L18-L21 — main (calls, depth 1)
    20: await buildGraph(dir, { contextDir, graphOnly: true, onlyDirs, extensions });
  • src/graph/refresh.ts:L202-L225 — rebuild (calls, depth 1)
    205: await buildGraph(args.dir, {
  • src/graph/workspace-cli.ts:L155-L172 — runWorkspaceGrep (calls, depth 1)
    161: const { result, coverage } = federateGrep(root, override, pattern, { ignoreCase: opts.ignoreCase, fixed: opts.fixed });
  • src/graph/refresh-run.ts:L1-L33 — refresh-run.ts (calls, depth 2)
    9: import { buildGraph } from "./build.js";
  • src/graph/refresh.ts:L271-L347 — ensureFreshGraph (calls, depth 2)
    287: // other repo gets the caller's own "no graph — run graft build" message.
  • src/graph/workspace-cli.ts:L65-L108 — buildChild (calls, depth 2)

Pull Request Review — 4 symbols in 4 files

  • src/app/review.ts:L45-L99 — reviewPullRequest (calls, depth 1)
    55: await buildGraph(checkout.dir);
  • src/app/review-process.ts:L178-L182 — childReviewer (references, depth 2)
  • src/app/review-worker.ts:L67-L87 — run (calls, depth 2)
  • src/app/server.ts:L31-L37 — AppSeams (references, depth 2)

MCP Tool Dispatch — 3 symbols in 1 file

  • src/mcp/tools.ts:L265-L351 — callSingleTool (calls, depth 1)
    288: const r = skeleton(root, file, { contextDir: dirOverride });
  • src/mcp/tools.ts:L157-L203 — callWorkspaceTool (calls, depth 1)
    185: const { result, coverage } = federateGrep(root, dirOverride, pattern, {
  • src/mcp/tools.ts:L233-L262 — callTool (calls, depth 2)

CLI Entry Point — 2 symbols in 2 files

  • src/cli.ts:L1-L3499 — cli.ts (calls, depth 1)
    4: * check, viz, mcp, callers, skeleton, grep, map, init. Git is the sync: commit
  • src/engine.ts:L104-L115 — graph (calls, depth 1)
    105: return buildGraph(dir, {
Test signal per changed area — 3 ✓

Reached = a node under a test path has a resolved edge into the changed symbol. It undercounts anything called indirectly — through a CLI, a spawned process or a dynamic import — so read a low ratio as “look here”, never as a coverage gate.

  • ✓ Workspace Graph — 2 of 2 reached · 2 test files changed here: test/ask.test.ts, test/generic-extract.test.ts
  • ✓ Grep Search — 2 of 3 reached · 1 test file changed here: test/search-grep.test.ts
    • not reached: formatGrepHeader
  • ✓ skeleton — 1 of 1 reached · 2 test files changed here: test/ask.test.ts, test/generic-extract.test.ts
51 test suites also reference this code

64 symbols, kept out of the diagram and the table so they cannot crowd out the areas a reviewer has to look at.

  • test/ask-index.test.ts
  • test/claude-hooks.test.ts
  • test/claude-prompt-private.test.ts
  • test/container-extract.test.ts
  • test/context-only-dir.test.ts
  • test/context.test.ts
  • test/covers.test.ts
  • test/depth-parity.ts
  • test/graph-bash.test.ts
  • test/graph-elixir.test.ts
  • test/graph-extensions.test.ts
  • test/graph-go.test.ts
  • test/graph-incremental.test.ts
  • test/graph-invariants.test.ts
  • test/graph-java.test.ts
  • test/graph-jsx.test.ts
  • test/graph-kotlin.test.ts
  • test/graph-languages.test.ts
  • test/graph-native-fallback.test.ts
  • test/graph-php-references.test.ts
  • …31 more

graft blast · origin/main...HEAD · depth 2 · 9 changed files

Open the interactive graph → — click an area to see its dependent symbols at file:line.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

grep / find_all / skeleton hide files that failed to parse: a miss is blamed on the pattern, not the index

1 participant