Skip to content

feat(release): refuse absent topics without a model and rank answers first more often in v6.5.0 - #220

Merged
jb-thery merged 1 commit into
mainfrom
release/v6.5.0
Nov 1, 2026
Merged

jb-thery merged 1 commit into
mainfrom
release/v6.5.0

Conversation

@jb-thery

@jb-thery jb-thery commented Nov 1, 2026

Copy link
Copy Markdown
Member

Release highlights:

  • get nothing instead of noise without any model: with local-hash, 65 of 90 off-topic and 27 of
    110 near-domain questions are refused on three real bases, up from 12 and 1, with every answer
    still in place
  • find the right file first more often on every measured index: from 44% to 54% on a
    30,171-chunk monorepo, and from 28% to 35% on the same monorepo without a model
  • see more off-topic questions refused with a semantic model: 68 of 90 instead of 54
  • search faster: median warm searches take 16 to 36 ms instead of 20 to 39 ms
  • embed document batches on the GPU with ingestEmbeddingDevice: "webgpu", which rebuilt this
    repository's index twice as fast on Apple silicon
  • reach GitHub and npm from buttons under the landing's terminal demo

Release details:

  • Ranking: function words weigh 0.3 in the keyword query, and passages whose file name or
    heading path holds words of the question gain up to 20%, ignoring words that more than half of
    the passages hold; the ranking policy moves to version 9
  • Abstention: with a semantic model, a keyword-only passage needs half of the question's IDF
    weight; local-hash answers nothing when its first five passages are weak or when words found
    nowhere in the index carry half of the question's weight; coverage credits fuzzy matches and
    identifier parts
  • Ingestion: ingestEmbeddingDevice runs document batches on the CPU or through ONNX
    Runtime's WebGPU provider, with a CPU fallback; queries stay on the CPU
  • Diagnostics: rgr doctor and rgr upgrade name an explicit rgr ingest when external
    extractors block repairs, and rgr evaluate times latency on sequential warm searches
  • Performance: query-independent work is memoized per passage text and file path
  • Benchmarks: the repository fixture moves the retired Chat and TTS questions to its
    near-domain list, and the observability gate checks phases by name
  • Landing: GitHub and npm buttons under the terminal demo and the measured hero proof
  • CI: the macOS portability job runs on macos-15, since GitHub retires macos-14 on
    November 2, 2026
  • Documentation: both READMEs, configuration, migration, CLI, troubleshooting, llms.txt, and
    the benchmark guide

Verification:

  • pass pnpm validate with Biome, Knip, 570 Core tests and 27 landing tests with coverage gates,
    API, CLI, MCP and public-surface smoke checks, builds, publint, and release checks
  • pass the pull request checks: Quality gate, Linux x64, macOS ARM64 and Windows x64 portability
    on Node 22, Commitlint, and CodeQL TypeScript analysis
  • measure 6.4.0 against this release on three real bases, each with its semantic model and with
    local-hash: no false refusal of 500 answerable questions, the right file first more often on
    all six indexes, and lower warm medians
  • append an unknown word to 250 answerable questions: semantic refusals keep their 6.4 behavior
  • verify the staging landing in English and French at 375 px and 1440 px in a browser

…first more often in v6.5.0

Release highlights:
- get nothing instead of noise without any model: with local-hash, 65 of 90 off-topic and 27 of
  110 near-domain questions are refused on three real bases, up from 12 and 1, with every answer
  still in place
- find the right file first more often on every measured index: from 44% to 54% on a
  30,171-chunk monorepo, and from 28% to 35% on the same monorepo without a model
- see more off-topic questions refused with a semantic model: 68 of 90 instead of 54
- search faster: median warm searches take 16 to 36 ms instead of 20 to 39 ms
- embed document batches on the GPU with `ingestEmbeddingDevice: "webgpu"`, which rebuilt this
  repository's index twice as fast on Apple silicon
- reach GitHub and npm from buttons under the landing's terminal demo

Release details:
- **Ranking:** function words weigh 0.3 in the keyword query, and passages whose file name or
  heading path holds words of the question gain up to 20%, ignoring words that more than half of
  the passages hold; the ranking policy moves to version 9
- **Abstention:** with a semantic model, a keyword-only passage needs half of the question's IDF
  weight; local-hash answers nothing when its first five passages are weak or when words found
  nowhere in the index carry half of the question's weight; coverage credits fuzzy matches and
  identifier parts
- **Ingestion:** `ingestEmbeddingDevice` runs document batches on the CPU or through ONNX
  Runtime's WebGPU provider, with a CPU fallback; queries stay on the CPU
- **Diagnostics:** rgr doctor and rgr upgrade name an explicit rgr ingest when external
  extractors block repairs, and rgr evaluate times latency on sequential warm searches
- **Performance:** query-independent work is memoized per passage text and file path
- **Benchmarks:** the repository fixture moves the retired Chat and TTS questions to its
  near-domain list, and the observability gate checks phases by name
- **Landing:** GitHub and npm buttons under the terminal demo and the measured hero proof
- **CI:** the macOS portability job runs on macos-15, since GitHub retires macos-14 on
  November 2, 2026
- **Documentation:** both READMEs, configuration, migration, CLI, troubleshooting, llms.txt, and
  the benchmark guide

Verification:
- pass pnpm validate with Biome, Knip, 570 Core tests and 27 landing tests with coverage gates,
  API, CLI, MCP and public-surface smoke checks, builds, publint, and release checks
- pass the pull request checks: Quality gate, Linux x64, macOS ARM64 and Windows x64 portability
  on Node 22, Commitlint, and CodeQL TypeScript analysis
- measure 6.4.0 against this release on three real bases, each with its semantic model and with
  local-hash: no false refusal of 500 answerable questions, the right file first more often on
  all six indexes, and lower warm medians
- append an unknown word to 250 answerable questions: semantic refusals keep their 6.4 behavior
- verify the staging landing in English and French at 375 px and 1440 px in a browser
@jb-thery
jb-thery merged commit de27f32 into main Nov 1, 2026
14 checks passed
@jb-thery
jb-thery deleted the release/v6.5.0 branch November 1, 2026 16:40
@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown

🎉 This PR is included in version 6.5.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant