Skip to content

feat: refuse absent-topic questions without a model, sharpen ranking, and move the hero buttons - #219

Merged
jb-thery merged 8 commits into
developfrom
feature/absent-focus-and-hero-layout
Nov 1, 2026
Merged

jb-thery merged 8 commits into
developfrom
feature/absent-focus-and-hero-layout

Conversation

@jb-thery

@jb-thery jb-thery commented Nov 1, 2026 •

Copy link
Copy Markdown
Member

Problem

Ragmir 6.4.0 still answered too many questions its sources cannot answer when no embedding model
is configured, and its ranking left room on real repositories:

  • with local-hash, only 12 of 90 off-topic and 1 of 110 near-domain questions were refused on
    three real bases;
  • a passage that only the keyword search returned could stand as an answer on one shared everyday
    word, and rare function words such as "what" or "de" could rank passages on their own;
  • on a small corpus whose headings share one word, a heading boost could bury the passage both
    retrieval channels ranked first, and the search then returned nothing;
  • rgr evaluate timed concurrent cases, so its latency read several times higher than real searches;
  • the landing hero showed the GitHub and npm buttons in the left column instead of under the
    terminal demo.

Changes

  • Ranking: function words weigh 0.3 in the keyword query; passages whose file name or heading
    path holds words of the question gain up to 20%, ignoring words that more than half of the
    passages hold. Ranking policy version 9.
  • Abstention: with a semantic model, a keyword-only passage needs half of the question's IDF
    weight. With local-hash, search answers nothing when none of the first five passages holds 30%
    of the weighted words, or when words found nowhere in the index carry half of the question's
    weight (a missing word weighs as if one passage held it). Coverage credits fuzzy matches and
    identifier parts.
  • Speed: query-independent work is memoized per passage text and path.
  • GPU ingestion: ingestEmbeddingDevice: "webgpu" embeds document batches on the GPU and falls
    back to the CPU; queries stay on the CPU.
  • Diagnostics: rgr doctor and rgr upgrade name an explicit rgr ingest when external
    extractors block automatic repairs; rgr evaluate takes latency from sequential warm searches.
  • Landing: the GitHub and npm buttons sit under the terminal demo, aligned right, and the hero
    proof shows the new measurements.
  • Docs: both READMEs, configuration, migration, CLI, troubleshooting, llms.txt, and the
    benchmark guide.
  • CI: the macOS portability job moves from macos-14, which GitHub retires on November 2,
    2026 and already fails during scheduled brownouts, to macos-15.

Measured results

Ragmir 6.4.0 against this branch on the same six indexes (three real bases, each with its semantic
model and with local-hash), two rounds, warm searches on an Apple M3 Max:

Measure 6.4.0 This branch
Answer first, Java/TypeScript monorepo (e5-base) 44% 54%
Answer first, same monorepo with local-hash 28% 35%
Off-topic questions refused 66 of 180 133 of 180
Near-domain questions refused 24 of 200 53 of 200
Correct answers refused 0 of 500 0 of 500
Median search, monorepo 39 ms 36 ms

The answer ranks first more often on all six indexes. One loss: local-hash on this repository
finds three fewer answers in the top 5. Ideas tested and rejected on the same indexes: file-level
rank fusion, joined query words, a test-file prior, score-based fusion, and the file path in the
embedded text.

Verification

  • pnpm validate: Biome, Knip, 570 Core and 27 landing tests with coverage gates, API, CLI, MCP,
    and public-surface smoke checks, builds, publint, and release checks
  • an absent-word stress test on 250 answerable questions: the semantic rule stays at its 6.4
    behavior, and local-hash refuses 2 more of the altered questions
  • pnpm bench:competitors: Ragmir reaches the top 3 for 83 to 88% of questions, against 63% for
    LangChain.js and 79% for MiniSearch
  • the hero checked in a browser at 1440 px and 375 px in English and French

…sharpen ranking

- weight function words such as "what", "de", or "les" at 0.3 in the keyword query, and lift
  passages whose file name or heading path holds words of the question by up to 20%
- ignore words that more than half of the passages hold in those boosts, so a heading word that
  every file shares no longer buries the passage both channels ranked first
- with a semantic model, require half of the question's IDF weight for a passage that only the
  keyword search returned, and credit fuzzy matches and identifier parts in coverage
- with local-hash, answer nothing when none of the first five passages holds 30% of the
  question's weighted words, or when words found nowhere in the index carry half of its weight
- weigh a missing word as if one passage held it, so one unusual verb on a small index does not
  pass for the question's focus
- memoize query-independent work per passage text and file path and read Arrow rows as plain
  objects, for faster warm searches
- raise the ranking policy to version 9
- add `ingestEmbeddingDevice` ("cpu" by default, or "webgpu"): Transformers ingestion then embeds
  document batches through ONNX Runtime's bundled WebGPU provider and falls back to the CPU when
  the runtime offers no WebGPU
- keep query embeddings on the CPU, where short inputs run faster, and key cached pipelines by
  device so CPU and GPU extractors never mix
…airs

- have `rgr doctor` and `rgr upgrade` recommend `rgr ingest` or `rgr ingest --rebuild` when
  configured OCR or legacy Word commands keep setup, repair, and upgrade from ingesting on their
  own, instead of a repair step that cannot ingest
- keep scoring cases concurrently, but take p50 and p95 latency from an untimed warm-up and up
  to ten sequential searches; concurrent cases waited behind each other and read several times
  higher than real searches
- move the questions about the retired Chat and TTS packages from the answerable list to the
  near-domain list of the repository fixture
- check observability phases by name, so adding or removing one fails visibly
- derive the runtime-reuse topK change from the balanced default, which setup leaves unset
…gestion

- update both READMEs, configuration, migration, CLI, troubleshooting, llms.txt, and the
  benchmark guide with the measured 6.4 to 6.5 results on six indexes and the stack comparison
- document the absent-topic rule, the distinctive-word boosts, and `ingestEmbeddingDevice`
- move the hero's View on GitHub and View on npm buttons to the right column, under the terminal
  animation and aligned with its right edge, on desktop and mobile
- show the measured 94% top-10 recall and 36 ms median search in the hero proof
- GitHub retires the macos-14 runner images on November 2, 2026 and fails their jobs during
  scheduled brownouts, which cancelled the macOS ARM64 portability job after one second
- macos-15 is the arm64 image the retirement notice recommends, so the job keeps its name and
  platform checks
@jb-thery
jb-thery merged commit 0a23897 into develop Nov 1, 2026
12 checks passed
@jb-thery
jb-thery deleted the feature/absent-focus-and-hero-layout branch November 1, 2026 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant