Pluggable CodeIndex interface: make any external indexing engine a first-class retrieval source, not just N mounted MCP tools

[saucam]

Code indexing has become a crowded category, and the engines are converging on one shape while diverging on license. codeoid should plug into whichever one the user configures, and — the actual point — fuse its output into recall rather than dumping its tools into the model's lap.

The landscape, surveyed

Engine	Stars	License	Parser	Graph store	Interfaces
GitNexus	45.1k	PolyForm Noncommercial	tree-sitter, 14+ langs	LadybugDB, .gitnexus/	MCP (17 tools), REST, CLI, web UI, library
gortex	1.1k	Apache-2.0	tree-sitter + compiler-grade, 257 grammars	gob+gzip snapshots, pluggable	MCP (100+ of 175 tools), /v1/* + SSE, CLI, web UI
axon	727	MIT	tree-sitter, Py/TS/JS	KuzuDB, .axon/kuzu/	MCP (axon_query/axon_context/axon_impact), FastAPI, CLI, web

Convergent: tree-sitter → structural knowledge graph in an embedded local DB, plus optional embeddings and hybrid BM25/vector, exposed over MCP + HTTP + CLI. All three are local-only with zero cloud dependency, which clears codeoid's self-hostable constraint.

Divergent, and decisive: the most popular engine in the category is PolyForm Noncommercial. We cannot bundle or depend on GitNexus commercially, but a user may absolutely want to use it. That is the argument for an interface rather than an adoption — and it is why #11 ("adopt a local code-graph MCP server") should become one implementation behind this seam rather than the plan itself.

What already works today — be honest about the baseline

Any of these can be mounted right now through the existing MCP registry as stdio {command,args,env} or http {url,headers}, and mcp/tool-source.ts surfaces it on every backend. So "plug in an index engine" is already possible at the crudest level.

What the MCP-mount route does not give you — this is the ticket

1. The model has to remember to call it. gortex exposes 100+ tools; GitNexus 17. Dumping that into the tool surface is precisely the tool-selection pressure problem tracked as open question fix: six confirmed audit findings (token lifecycle, msg-drop, recall scaling) #12 in conductor-design.md, and it degrades badly on a small open-weight backend. A retrieval source is consulted by the engine; a tool is consulted by the model's judgment. Only the first is reliable.
2. No fusion. RecallHit.components is {vector, fts, recency, pathOverlap} combined by a weighted sum in ranker.ts. A structural score cannot participate, so you end up with two separate retrieval systems instead of one better one.
3. No lifecycle. No freshness signal, no incremental update on file change, no status surface. IndexScheduler already does exactly this job for codeoid's own index.
4. No governance. An opaque mount can't satisfy the mount contract (Compose the conductor system prompt per mounted capability (and stop keeping it in fleet.ts) #278) — no declared read set means no approval classification and no episodic capture.
5. No workspace-index contribution. codeoid injects a workspace memory index into every system prompt. An engine's architecture summary and cluster labels are exactly what belongs there, and a tool mount cannot contribute to it.

Why this is strategically ours, not catch-up

#282 established that codeoid's retrieval moat is the corpus — tool-call-granular episodes with file paths — because a text-ingesting memory layer never sees a tool call. A code index is the complementary axis: static structure (who calls what, what type is this, what is dead) against behavioral history (what agents actually did to these files, together, when).

The sharpest instance: axon computes change coupling from git history. #69 computes file co-occurrence from agent tool calls. Those are different couplings — what humans historically changed together, versus what agents touched together in sessions. Fusing static graph + git coupling + agent coupling produces a signal no engine in the table above can compute, because the third term requires the episode corpus. That is the differentiator, and it only exists if the index is a fused source rather than a mounted tool.

Proposed design

The interface — mirror Embedder/Reranker

Small, factory-constructed, registered like a provider with markUnavailable(hint) when the binary or service is unresolvable (the pi resolution pattern in providers/registry.ts is the model).

export interface CodeIndex {
  readonly id: string;
  readonly displayName: string;
  init(): Promise<void>;
  /** Optional capabilities this engine actually supports — probe, never assume. */
  capabilities(): CodeIndexCapabilities;
  /** Freshness + coverage, so recall can decide whether to trust it. */
  status(workspaceId: string): Promise<CodeIndexStatus>;
  /** Ensure/refresh; incremental where the engine supports it. Driven by IndexScheduler. */
  sync(workspaceId: string, changed?: string[]): Promise<void>;
  /** The retrieval primitive that fuses into the ranker. */
  search(q: CodeQuery): Promise<CodeHit[]>;
  /** Graph primitives, guarded by capabilities(). */
  neighbors?(node: CodeRef, edge: CodeEdgeKind, depth?: number): Promise<CodeRef[]>;
  impact?(node: CodeRef): Promise<CodeRef[]>;
  close(): Promise<void>;
}

Capability probing, not a fat interface. The engines differ wildly — 257 grammars with compiler-grade resolution versus three languages, and completely different graph vocabularies. A fat required surface would exclude nearly everything. Required: search + status + sync. Everything else optional behind capabilities(), with graceful degradation and never an error when unsupported.

Normalize into a canonical CodeHit/CodeRef, the same discipline providers/canonical.ts applies to provider events. That normalization is what makes fusion possible at all.

Two tiers, exactly like the backend strategy

COMPARISON.md already explains why codeoid runs native providers plus a cheap ACP adapter rather than a PTY tier. Same shape here:

• Native adapter per engine — full fidelity, uses the engine's library or typed HTTP API.
• Generic MCP-backed adapter — since every engine in the table ships MCP, one adapter plus a config mapping of tool names to interface methods covers the whole category cheaply. This is the "any engine the user configures" answer.

Three surfaces for the results

1. Fused ranker signal. Add a codeStructure component to RecallHit.components behind a weight that defaults to 0 when no index is configured, so existing behavior is bit-identical without one.
2. A small stable tool set of ours — roughly code_search / code_context / code_impact — normalized across engines, so a backend sees ~4 stable tools instead of 100 engine-specific ones. This is what fixes the tool-count problem rather than inheriting it.
3. Workspace-index contributor — architecture summary and cluster labels into the injected system-prompt index.

Scope

• CodeIndex interface + CodeIndexCapabilities + canonical CodeHit/CodeRef types
• Registry with resolution + markUnavailable(hint), config under codeIndex.* (engine id, command/url, weight, sync cadence)
• Generic MCP-backed adapter with a tool-name mapping
• One native adapter as the reference — gortex (Apache-2.0) is the right first pick: permissive license, single static binary with zero external deps, widest language coverage, and a versioned /v1/* API
• Ranker fusion behind a default-0 weight
• IndexScheduler drives sync() out-of-band — gortex indexes the Linux kernel (70k files) in ~3 min, so this must never touch a turn's critical path
• Normalized code_* tool set replacing raw engine tool exposure
• Mount contract compliance (Compose the conductor system prompt per mounted capability (and stop keeping it in fleet.ts) #278): declared read verbs, identity attribution, audit, episodic capture
• Adapters record the engine's license so a commercial deployment cannot accidentally take a PolyForm-NC dependency

Open questions

• Worktree interaction. Engines write .gitnexus/ / .axon/ into the repo, but codeoid anchors memory on git-common-dir precisely so worktrees share it. Per-worktree indexes would duplicate a multi-minute build. Policy needed.
• Tenancy. The index is per-workspace on disk, not per-(account, project). Probably acceptable for local single-operator use; state the boundary rather than discover it.
• Trust. An index engine is a third-party binary that reads every file in the repo. Same class of problem as registry-level pack trust (Registry-level trust: skill linking is per-registry but trust is per-pack #234), and it deserves the same answer.
• Do we need the graph at all, or only search? A v1 that fuses search and ignores neighbors/impact is far cheaper and may capture most of the value. Worth measuring against the P0 fixture before building graph plumbing.

Related

• feat: adopt a local code-graph MCP server (CodeGraph) for code-aware retrieval #11 adopt CodeGraph MCP — becomes one implementation behind this interface
• Retrieval strategy: build the corpus-derived signals, evaluate a library for the ranker layer #282 retrieval strategy — this is the second axis of the Tier B corpus argument
• feat: file co-occurrence graph — 1-hop neighbor expansion after initial recall to surface related episodes #69 file co-occurrence graph — the agent-coupling term that makes fusion differentiated
• Compose the conductor system prompt per mounted capability (and stop keeping it in fleet.ts) #278 mount contract — how an index earns auto-approval
• Registry-level trust: skill linking is per-registry but trust is per-pack #234 registry-level trust — the analogous third-party trust problem