Skip to content

Retrieval strategy: build the corpus-derived signals, evaluate a library for the ranker layer #282

Description

@saucam

Strategic split of the open retrieval backlog, from the market review behind #275 and now recorded in conductor-design.md §6. Not a proposal to slow retrieval down — it serves both the conductor's session resolution and every session's own recall. It is a proposal about which half we author.

The finding

Zep/Graphiti ships, benchmarked and published, the exact stack several of these issues re-implement: bi-temporal edges with explicit t_valid/t_invalid validity intervals, hybrid retrieval (cosine + BM25 + graph traversal), RRF / MMR fusion with optional cross-encoder scoring, ~300 ms P95 with no LLM call on the retrieval path, 20k+ stars, DMR numbers in the literature.

So framing "hybrid + bi-temporal + rerank" as codeoid's moat invites a comparison codeoid loses on evidence, and hand-rolling it is months of work to reach library parity. §6 of the design doc previously did exactly that framing; #275 corrects it.

The split

Tier A — commodity plumbing. Evaluate adopting rather than authoring.

Every one of these is a known-good technique with a maintained implementation. Worth an explicit build-vs-adopt call before writing more of it, weighed against the constraint that actually matters here: codeoid must stay self-hostable with no external service (bun:sqlite + WASM embeddings). A library that needs Neo4j fails that test, which may well be the deciding argument for continuing to author — but let's decide it on that basis rather than by default.

Tier B — corpus-derived. Nobody else can build these. Prioritise them.

These consume signals that exist only because of the native-protocol provider layer: file paths per tool call, session-to-session edges, per-episode access counts, tool-call granularity. A memory layer that ingests text cannot produce them, because it never sees a tool call. Graphiti's graph is entities extracted from prose; #69 and #70 are structural graphs derived from code artifacts and session topology. That is not a gap a competitor closes by shipping a better ranker.

Which is a satisfying result: the native-protocol decision that costs 300–1,300 LOC per backend pays off a second time, in retrieval.

The other differentiator is the task, not the stack

"Fuzzy human reference → the right session across N workspaces" is not fact recall, and no incumbent publishes a number for it. DMR measures something else. The P0 labeled fixture (37 genericised fuzzy references over the real 11-workspace corpus, ~22% naive P@1 baseline, ~73% after global fusion + cross-encoder) is measuring a task the field does not benchmark.

That makes it a publishable result, not only an internal go/no-go gate — and publishing it is worth more competitively than shipping a faster ANN index.

Asks

  • Explicit build-vs-adopt decision on Tier A, with the no-external-service constraint stated as the deciding criterion
  • Re-prioritise Tier B above Tier A in the backlog
  • Grow the P0 fixture toward publishable size and write it up
  • Update any positioning (README, COMPARISON.md) that credits the ranker rather than the corpus

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions