Repository navigation
feat: calibrate abstention per index, compare with other stacks, and rebuild the landing drawer - #217
Merged
jb-thery merged 5 commits intoNov 1, 2026
Conversation
…x health - measure each Transformers index's noise floor at ingest from 24 off-topic probe questions, store it in the manifest, and measure it once per process for older indexes - treat a passage as a strong vector match only when it beats the noise floor, and answer nothing when the first five semantic results are all weak - flag mxbai-embed-xsmall passages as weak below 30% of the question's words instead of 50%, which flagged a third of the correct answers on a code repository - select `_score` in term-frequency queries so LanceDB stops printing a deprecation warning on every search - throw CONFIG_MIGRATION_REQUIRED instead of an internal error for configs with retired fields - have doctor ask for `rgr storage optimize` on fragmented indexes and for `rgr ingest` when a semantic index has no stored noise floor - document the noise floor, weak answers, the migration notes, and the new error code
- share one nearest-rank percentile in Core and one in the benchmark library instead of eight copies - read JSON Lines files through one helper for the ingestion snapshot and the fingerprint cache, and document why the append-only journal tolerates a torn last line - validate the ingestion run state through the run metadata check instead of a copy of it - delete the unused staged-manifest writer and fallback gitignore entry, unexport internal constants and helpers, and mark the deprecated command aliases - render the 404 and 500 pages from one error-page component that reuses the logo component
…and gate on Knip - add `pnpm bench:competitors`: the same tracked files, questions, embedding model, and vectors for every stack, with off-topic and near-domain refusals and latency - add the public question fixture: 99 developer questions with every answering file, 30 off-topic and 30 near-domain questions - add Knip to `pnpm validate` with entries for benchmark workers and release scripts, declare TypeScript at the root for the API check, and document the rule for contributors
… drawer - add View on GitHub and View on npm buttons under the hero calls to action - rebuild the mobile drawer as cards of icon rows with chevrons, Documentation and Project sections, language pills, the version, and pinned GitHub and npm buttons - list every project link from one module shared by the hero, drawer, footer, and structured data, adding the FAQ, benchmarks, and AGPL license links that some surfaces missed - show the measured 39 ms median on the 29,778-chunk monorepo in the hero proof
- lead both READMEs with the current measured recall and latency of the monorepo index - show Ragmir 6.4 against 6.3.1 on three real bases: same answers, more refusals of off-topic and near-domain questions, fewer correct answers flagged as weak - add the comparison with LangChain.js, Orama, and MiniSearch on this repository
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Using Ragmir on eleven real local bases showed that it rarely says "I found nothing":
returned passages for nearly every off-topic question (0 of 30 refused on a monorepo);
weakEvidence;_scoredeprecation warning, configs with retired fields failed withan
INTERNALerror, and indexes left in hundreds of small fragments searched five times slowerwithout any advice.
The benchmarks also never compared Ragmir with other retrieval stacks, and the landing had no direct
path to GitHub or npm in the hero and a drawer that missed several links.
Changes
questions (older indexes measure it once per process). Only passages nearer than that floor count as
strong vector matches, and a semantic search whose first five results are all weak answers nothing.
mxbai flags weak evidence below 30% of the question's words instead of 50%. Term statistics select
_score, retired configs throwCONFIG_MIGRATION_REQUIRED, andrgr doctorasks forrgr storage optimizeon fragmented indexes and forrgr ingestwhen no floor is stored.pnpm validate. Eight copies ofpercentile, two JSON Linesreaders, a duplicated run-state validator, and the 404/500 pages now share one implementation; dead
code is gone and internal symbols are no longer exported.
pnpm bench:competitorscompares Ragmir with LangChain.js, Orama, and MiniSearch onthis repository with the same files, questions, model, and vectors.
with Documentation and Project sections, and one shared module for every project link.
llms.txt, contributor guides, and both READMEs.Measured results
Ragmir 6.3.1 against this branch, same indexes, warm searches on an Apple M3 Max:
No false refusal on 260 answerable questions. On this repository, the answering file reaches the top 3
for 76% of questions with Ragmir, 56% with LangChain.js, and 70% with MiniSearch.
Validation
pnpm validate: Biome, Knip, audit, type checks, 556 Core tests and 27 landing tests with coveragegates, builds, API check, CLI/MCP/public-surface smoke, publint, release smoke and artifacts
Escape and focus return, Team row, footer links)
Limitations
in each process until the next
rgr ingest.with the corpus.