Skip to content

fix(search): cold FTS build out of 2s budget plus index experiment-output guard - #62

Merged
ManSio merged 8 commits into
mainfrom
fix/fts-cold-plus-index-guard
Sep 29, 2026
Merged

ManSio merged 8 commits into
mainfrom
fix/fts-cold-plus-index-guard

Conversation

@ManSio

@ManSio ManSio commented Sep 29, 2026

Copy link
Copy Markdown
Owner

Cherry-pick of 5a47602e + bff72e2f onto current main (parallel PRs #53/#54 already merged; original PR #52 branch conflicting).

  • Cold FTS to_pandas build measured 2.47s > 2.0s wait_for: first search in a fresh process silently dropped the FTS tier (flaky A/B 18/20 vs 8/20). Build hoisted out of timeout (idempotent, locked).
  • _get_ext_dir off-by-one (3 parents instead of 4): dev-mode branch dead, models resolved to empty data_root.
  • Experiment outputs (experiments/**/results|work, measured 3127/15426 chunks = 20.3 pct garbage) excluded from index via new SystemArtifacts.is_experiment_output layer; git tracking untouched (section 17).

Verified: 32 targeted tests green, ruff clean. Takes effect on purge + reindex (follow-up).

MSCodeBase Agent added 2 commits September 29, 2026 15:04
Cold to_pandas build measured 2.47s > 2.0s wait_for: first search in a
fresh process silently dropped the FTS tier (flaky A/B 18/20 vs 8/20).
Build hoisted out of timeout (idempotent, locked); 2s kept for search.
_get_ext_dir took 3 parents instead of 4: dev-mode branch dead, models
resolved to empty data_root. Embedder GGUF copied to git-ignored models/.
3127/15426 chunks (20.3 pct) were experiments slash results|work dumps
(ctx files up to 329 chunks each) choking lexical search (P2) and cold
FTS build (2.47s). New SystemArtifacts.is_experiment_output layer wired
into FileGuard; frozen inputs and experiment sources stay indexed.
Takes effect on full reindex. Guard: test_experiment_output_guard.
@coderabbitai

coderabbitai Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: bb23e3ba-97e7-4208-b33c-2d1d89defc5c


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

MSCodeBase Agent added 5 commits September 29, 2026 15:20
772 files / 2152 chunks under experiments slash results|work removed via
delete_file plus graph cleanup plus compaction (prune safety-guard at
50pct would refuse: garbage was 52.4pct of files). Verified 0 remaining.
Dry-run by default.
RU B 26/80 (32.5pct) vs EN B 30/80 (37.5pct), same clean index, same
stack: CIs overlap, no significant language effect. 6/16 queries flip
per-language (all-or-nothing swaps, e.g. F5S-02 5-0): language changes
WHICH queries pass, not how many. Harness gains --queries-file (variants
live outside frozen/).
Run scratch (ctx/cand dumps, opencode.json) is reproducible from
judged_raw.json; keep only raw results plus queries plus aggregate.
No language effect (RU 32.5pct vs EN 37.5pct, overlapping CIs);
6/16 queries flip all-or-nothing. Purge 772 files verified 0 left.
PR52 closed as superseded by 62/63. Objective baseline stale.
Post-purge (772 garbage files removed): A hit@1 4/16, hit@3 5/16,
hit@10 6/16 (unchanged), B top-1 4/16. Top shifted by 1 query, within
n=16 noise: purge did not move the top significantly. Old 09-26 numbers
kept below as stale.
@ManSio
ManSio merged commit 3841b55 into main Sep 29, 2026
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant