diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 142ae2d..cdc49d3 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -101,6 +101,10 @@ CODEOID_COMPRESS_EXCLUDE= # comma-separated cmd prefixes to skip CODEOID_COMPRESS_PIPES=0 # allow compressing piped commands CODEOID_COMPRESS_MIN_BYTES=1024 # skip compression below this size +# Advisory guards (see FEATURES.md → Guards) +CODEOID_GUARD_REPEAT_TOOL=1 # loop-breaker advisory; on by default +CODEOID_GUARD_REPEAT_TOOL_EXCLUDE= # comma-separated tool patterns to ignore + # Auto-rotation (Layer D) CODEOID_AUTO_ROTATE=0 # auto-rotate backing session near context ceiling CODEOID_AUTO_ROTATE_WARN_PCT=0.75 # warn at this occupancy (no action) diff --git a/docs/FEATURES.md b/docs/FEATURES.md index 9d7c435..333ef97 100644 --- a/docs/FEATURES.md +++ b/docs/FEATURES.md @@ -330,6 +330,48 @@ The session auto-approves up to 50 write/exec actions. Reads + greps + memory re Status bar shows live budget: `autonomous (37 actions left)`. You can interrupt anytime with `Ctrl-X`. +### Guards + +A *guard* observes the session and may inject model-facing advice. +It never vetoes a tool call, never rewrites arguments, and never appears in the tool list — anything that needs to *stop* a call belongs to the approval flow or the autonomous budget instead. + +**Repeat-tool guard** (on by default) is the loop-breaker. +It counts runs of consecutive calls to the same tool with identical arguments, and at `3`, `5`, and `8` in a row it injects an escalating advisory telling the model to re-read the result it already has and either change approach or conclude. + +This matters most where nobody is watching. +An unattended `dispatch` worker wedged on `Read(same file)` or `Bash(same failing command)` will otherwise burn its whole tool budget and report failure with no diagnosis; the budget caps the damage, the guard catches the cause while the turn can still recover. + +Chains are tracked **per emitting agent**, so two subagents hammering the same tool in parallel are two independent runs rather than one interleaved chain that resets forever and never fires. +Argument comparison is order-insensitive (deep key-sort), so `{a:1, b:2}` and `{b:2, a:1}` are the same call. +Any inbound message — owner, background-task digest, or dispatch task — resets every chain, because the guard only claims "N identical calls with *nothing else happening*". + +```jsonc +// ~/.codeoid/config.json +"guard": { + "repeatTool": { + "enabled": true, + "thresholds": [3, 5, 8], // run lengths that fire; each must be >= 2 + "include": [], // patterns to track; empty = all tools + "exclude": ["TodoWrite", "todo_write"], + "argumentsPreviewChars": 500 // caps the reminder text, never detection + } +} +``` + +Invalid thresholds fail loud at construction rather than silently reverting to defaults — a guard that never fires because of a typo is worse than no guard. If the config is bad the session logs it and starts without the guard; an advisory plugin is never a reason to refuse a session. + +#### Model Experience + +The advisory arrives as injected context on the model's **next** request, wrapped in a `` block and explicitly labelled as daemon-authored so it is never mistaken for owner input. The tool call that triggered it is unaffected — already recorded, already on its way to approval or execution. + +Injection uses `later` priority, which merges into the running turn without starting a fresh query, so a reminder costs no extra turn. Backends without mid-turn injection get no reminders rather than a message arriving out of context. + +#### KV Cache effect + +Append-only. The advisory lands after the reusable request prefix and invalidates no prior cache entry. Cost is the advisory itself: ~60 tokens for the brief form, and up to `argumentsPreviewChars` more for the detailed form. + +> The **Model Experience** / **KV Cache effect** headings are a convention borrowed from DeepSeek Harness (see [prior-art-deepseek-harness.md](./prior-art-deepseek-harness.md) §4.9). Any feature that changes what the model sees should state both: what reaches the model, and whether it invalidates the prompt prefix. For a harness, prefix stability *is* cost. + ### Web UI Mobile-first SolidJS SPA at `http://localhost:7400/ui/`. Also works as a Telegram Mini App: diff --git a/docs/prior-art-deepseek-harness.md b/docs/prior-art-deepseek-harness.md new file mode 100644 index 0000000..a0b35fa --- /dev/null +++ b/docs/prior-art-deepseek-harness.md @@ -0,0 +1,339 @@ +# Prior art: DeepSeek Harness (`dsh`) + +> Analysis date: 2026-08-14. +> Read against the `deepseek-harness` tree at commit `47f9438` (`0.1.0-rc.5`, developer preview). +> Companion to [COMPARISON.md](./COMPARISON.md), which covers Superset and Omnigent. + +## 0. Measurements + +| | Codeoid | DeepSeek Harness | +|---|---|---| +| Non-test TypeScript | ~50k LOC (`src/`) + web + 2 packages | ~520k LOC across ~50 package groups (~270 packages) | +| Test files / test LOC | 195 / 52.7k | 647 / 217.5k | +| Package READMEs | — (monolithic `src/`) | 268, of which 215 carry a **Model Experience** + **KV Cache effect** section and 220 carry **Known Limitations and Deferred Work** | +| Decision records | `docs/*.md` design docs | 1,386 files under `.agents/notes/` | +| Generated + CI-verified catalogs | none | cordis surface, config, persistence, tool, module graph, scoped events — each with a `--check` gate | +| Licence | — | MIT | + +The size gap is real but it is not the interesting part. +The interesting part is that dsh spent its size on *seams* and codeoid spent its size on *features*, and that difference is now visible in both codebases. + +--- + +## 1. These are not the same category of thing + +This has to come first, because most of the feature-by-feature comparisons below are meaningless without it. + +**dsh is a harness.** +It implements the agent loop itself: turn/step orchestration, prompt assembly, the tool registry, the model adapter seam, compaction, the session log. +Its LLM adapters are `llm-deepseek` (the official wire format) and `llm-pi-ai` (a catalog-backed second path). +It is a single-user local product: a web UI on `127.0.0.1:3080`, a headless one-shot runner, and an ACP server. + +**Codeoid is a control plane over harnesses.** +It does not implement an agent loop. +It drives the Claude Agent SDK, the Codex app-server, the Gemini CLI over ACP, and three API-level backends, behind one `SessionProvider` interface, and adds identity, memory, multi-frontend attachment, and device handoff on top. + +The two do overlap at the edges, and the overlap is instructive. +dsh ships subagent providers for Claude Code, Codex, and ACP — so it *delegates* to other harnesses even though it doesn't *run as* them. +dsh also ships hook-compatibility bridges that execute a user's existing Claude Code and Codex hook configs on dsh's own interception points. +Both are convergence toward the same insight codeoid started from: the other harnesses are not going away, so interoperate with them. + +The difference that remains is directional. +dsh reaches out to other harnesses from a position of owning the loop. +Codeoid reaches out from a position of owning the session. +You cannot fork a dsh session onto Claude Code; you can fork a codeoid session onto Codex. + +--- + +## 2. Where codeoid genuinely stands out + +### 2.1 Identity — not close + +This is the widest gap in the comparison, and it runs in codeoid's favour. + +dsh's entire identity subsystem is one package, `dsh-anonymous-user-id`: a random UUID v4 written to `~/.dsh/.anonymous-user-id`, used to correlate telemetry, tag `/feedback` acknowledgements, and set an `x-deepseek-harness-user-id` header on DeepSeek requests. +Its own README states the identity is deliberately not derived from anything stable and resets when the file is deleted. +There is no authentication, no authorization, no scopes, no multi-user model, no delegation, no revocation, and no audit attribution anywhere in the tree. + +Codeoid has ZeroID JWT verification, eight enforced scopes checked per message, a cryptographic identity per agent *and* per sub-agent as SPIFFE/WIMSE URIs, delegated tokens with scope attenuation, cascading revocation, and a SQLite audit log attributing every action to a subject. + +That is not a gap dsh can close by writing a package. +It is a gap that follows from being a single-user local tool, and closing it would mean rebuilding the tool as a multi-tenant service. + +### 2.2 Cross-session memory — materially deeper + +dsh's recall story is `session-query`: a query vocabulary over the logical session corpus with a SQLite **FTS5** provider. +It is exact-read, relationship tracing, and full-text keyword matching over session logs. +There are no embeddings, no vector index, no reranker, and no clustering anywhere in the tree. + +Codeoid's `src/daemon/memory/` is FTS5 **plus** a transformers.js embedder, a transformers.js reranker, a hybrid ranker, a chunker, topic clustering with an LLM cluster-labeler, a workspace clusterer, an auto-regenerated workspace memory index injected into the system prompt, and an MCP surface exposing `recall` / `recall_file` / `timeline` / `get_episode` to the agent itself. + +Keyword search over your own logs and semantically-ranked verbatim episodic recall injected into context are different products. + +### 2.3 Multi-backend at the session level + +Codeoid runs six backends as first-class session drivers with cross-backend fork carrying a history seed. +dsh runs one loop against DeepSeek models; other harnesses appear only as *children*, and only through the subagent seam. + +This is the position codeoid should defend hardest, because it is the one that cannot be retrofitted. + +### 2.4 Multi-frontend and device handoff + +Codeoid: TUI (Ink, plus the native Rust client in `codeoid-ui`), web, Telegram, and mobile push, all attaching to the same daemon-owned session with scrollback replay on handoff. +dsh: a web UI, a headless runner, and an ACP server. +There is no phone story and no handoff story. + +### 2.5 Semantic pre-entry compression + +Codeoid's `src/daemon/compress/` has declarative, pure, unit-testable rules that *understand* the commands they compress — dedicated rules for git, test runners, search, and shell, ordered by specificity, with a tee-cache so the full output stays recoverable through recall. + +dsh's nearest equivalent, `spill`, is deliberately dumber and deliberately later: after a tool returns, if the result exceeds `maxInlineBytes`, persist the whole thing to a private session-scoped file and hand the model a head/tail preview plus an opaque locator and a retrieval hint. + +These are complementary rather than competing — see §4.5. + +### 2.6 The conductor / fleet layer + +A privileged supervising session with a strict read/send tool split, side-effecting verbs that can never be auto-approved, digest-based summaries so the supervisor never goes out of context, and a durable dispatch queue with lease reclaim and crash recovery. + +dsh has rich *intra*-session orchestration (§3.6) but nothing that supervises a population of independently-owned sessions across workspaces. + +### 2.7 Per-turn economics + +Persistent per-turn token / cost / cache telemetry and live context occupancy. +dsh has `ctx.tokenMeter` for estimation and replay, wired into compaction pressure decisions — but it is a compaction input, not a user-facing economics surface. + +--- + +## 3. Where dsh is clearly better + +### 3.1 Capability seams — the biggest architectural gap + +dsh's organizing idea is that a capability is three roles: a **Service Definition** declaring the interface, a **Service Provider** implementing it, and a **Consumer** using it — usually a model-facing tool. +A package may combine roles, but one role alone is not a seam, and adding a capability means designing all three. + +The payoff is stated plainly in their architecture doc and it holds up in the tree: + +> Filesystem and subprocess providers share one execution world, so pointing them at a remote sandbox moves Bash, PTY, and LSP with them, with no provider forks. + +That is why `e2b/` is three small packages (`e2b`, `fs-e2b`, `subprocess-e2b`) rather than a fork of the execution stack. + +Codeoid has exactly one real seam — `SessionProvider` — and it is a good one; the multi-backend position rests on it. +Everything else is direct. +The measurable consequence is `session-manager.ts` at **5,287 lines** and `session.ts` at **4,527 lines**: two files holding rate limiting, resume, scope enforcement, retry, scrollback, permissions, hooks, memory wiring, compression wiring, dispatch, and fleet. +Those two files are the main thing standing between codeoid and its next ten features. + +### 3.2 One append-only log as the single source of truth + +dsh has one durable structure — the session event log — and everything is a projection of it. +Model history comes from `deriveMessages()`. +Resume, fork, transcripts, telemetry, persistence, and the UI all derive from the same stream. +Events are classified `current` / `shadowed` / `log-only`, so replaced context and never-model-visible bookkeeping stay in the log without reaching the model. + +It is enforced, not merely intended: + +> **Model-visible means logged.** Anything that reaches a model request must be reconstructable from the log, and a runtime invariant asserts it. + +Plan mode is the clean demonstration: `plan/mode` is a log-only event, and `foldPlanMode(events)` is a pure fold, so resume, fork, and compaction all recover plan state with **no live mirror** to keep in sync. + +Codeoid has four overlapping representations — the JSONL `TranscriptStore`, the `ScrollbackBuffer` ring, the SQLite `Store`, and each provider's own native session file. +`src/daemon/resume-reconcile.ts` exists precisely because they can disagree: tool calls frozen in `streaming` / `waiting_confirmation` / `executing` are driven by in-memory state that does not survive a restart, so on resume they must be rewritten to a terminal phase or clients replay phantom running tools forever. + +That file is a well-written fix for a problem that the log-plus-fold architecture does not have. + +### 3.3 Compaction as a real subsystem + +dsh's compaction is a seam with a basic provider, and the rigor is well beyond codeoid's rotation: + +- A log-recorded lock bracketing the whole operation (`compaction/start` … `compaction/end`), ordered so that a crash mid-operation leaves a *detectable orphaned lock* rather than a false "finished" record. +- Range boundaries validated by `toolPairingBalancedBefore/After()` so an assistant tool call is never separated from its result — the exact thing that produces provider 400s. +- A separate `ctx.toolResultPruner` that does deterministic head/middle/tail pruning of oversized tool results *before* range selection, measured in Unicode code points so a retained boundary cannot split a surrogate pair, and re-measured through the token meter afterward. +- A named failure taxonomy (`busy` / `cancelled` / `changed` / `summary` / `commit` / `persistence`) where failed attempts stay visible in the log. +- Full reconstructability: the summarize call's provider, model, and cap are logged so the one-shot request can be rebuilt from log plus code. + +Codeoid's auto-rotation is arguably the better *idea* — lossless via a memory recall seed beats a lossy summary — but dsh's is the better *engineering*. + +### 3.4 OS-level sandboxing + +dsh ships `sandbox-local` with Linux bwrap/Landlock (including their own `node-addon-landlock-run` native addon), macOS Seatbelt, and a Windows ACL restricted-token backend. + +Three details worth stealing regardless of implementation: + +- **Enforcement is a reported fact, not an assumption.** `SandboxEnforcement` is `full | partial`; older Landlock ABIs and the Windows ACL runner's Everyone/hard-link boundaries report `partial`, and consumers requiring an absolute boundary must reject rather than treat it as full. There is a postmortem on getting this wrong (`0004-landlock-partial-notice-misclassified-child-failures`). +- **Denial signatures are per-backend dialects**, not a cross-backend union — EROFS under bwrap, EACCES under Landlock, EPERM under Seatbelt — because a union claims denials a given backend never produces. +- **`runnerFailureRules` separate "the sandbox refused to start" from "the sandbox worked and blocked the command."** Checked first, and surfaced as infrastructure failure rather than task failure. + +Silent unconfined passthrough is never legal for a confined policy. + +Codeoid has approvals and an autonomous write budget, which are a policy layer, not a confinement layer. +For a product whose thesis is *identity-first*, this is a soft spot: perfect attribution of an action you did not confine tells you exactly who deleted your home directory. + +### 3.5 LSP as a model-facing capability + +Four operations — `goToDefinition`, `findReferences`, `goToImplementation`, `hover` — behind `ctx.lsp`, with a generic stdio provider and a model-facing tool. +The union is deliberately closed so adding an operation is a compile-enforced change across seam, providers, and tool. + +Codeoid has none. +For a coding harness this is a real capability gap, and it is a bounded one. + +### 3.6 Continuable background subagents + +Codeoid's subagent story is lineage tracking (`parentToolUseId`) plus a dispatch queue. +dsh's is a full lifecycle: + +- One-shot children *and* **continuable** children that persist, cold-resume from their own session, and accept later follow-ups. +- Follow-up authority checked against the exact live direct parent recorded in the child's durable header — checked before reconstruction and *again* in the final inbox-admission span, so a parent unregistered mid-materialization cannot authorize delivery. +- `listChildren()` / `listDescendants()` walking the whole session tree in stable pre-order *without loading or resuming* any of it. +- `drainContinuableDescendants()` closing admission below a parent and releasing the forest child-first. +- Provider capabilities (`outputSchema`, `depthLimit`, `toolFilter`, `persona`) declared so unsupported requests are rejected *before* child creation. +- `delegationDepth` persisted on the header and monotone, so a resumed child cannot be re-counted as top-level. +- `applyChildComposition(childCtx, parent, composition)` — one call that makes composing a child *without* joining the parent's preset unrepresentable at the call site. That is a beautiful way to encode an invariant. + +### 3.7 Guard plugins — small, and codeoid has neither + +- **`repeat-tool-reminder`**: watches each agent's tool stream, keys a chain on `(tool name, deep-key-sorted JSON of arguments)`, and at configured run lengths (`[3, 5, 8]`) injects an escalating advisory to stop repeating, re-read the last result, and change approach or conclude. It never vetoes and never appears in the tool list. The chain key always compares the full canonical string; the preview cap bounds only the reminder text. +- **`timeout-policy`**: one `tools/execute` around-listener arming a cooperative deadline from the tool's own declared `ToolDefinition.timeoutMs`. Zero config, so a mistyped tool name is not possible. + +Both are a few hundred lines and both target exactly the failure modes of long unattended runs — which is codeoid's dispatch and conductor use case. + +### 3.8 Engineering discipline treated as infrastructure + +The thing that most distinguishes this repo is that its process is *mechanized*, not documented-and-hoped-for: + +- **Every generator has a `--check` mode wired into a CI gate** — cordis surface catalog, config catalog, persistence catalog, tool catalog, module graph, scoped events, third-party notices. +- **`verify-type-equiv`** checks that the ` ```ts type-equiv ` blocks in the docs still match the source declarations they claim to mirror. Documentation drift becomes a build failure. +- Named gate groups (`ci-static`, `ci-primary`, `ci-coverage`, `ci-snapshot`, `ci-artifacts`, `ci-consumers`) plus `doc-budgets`, `doc-refs`, `verify-md-links`, `verify-md-wrap`, `agent-note-format`, `agent-note-classification`. +- **Windows as a blocking CI tier**, with wine-based gates and separate blocking / complete / observational levels. +- `knip` (dead code), `publint` (package correctness), `jscpd` (duplication), workspace constraint checks, runtime-closure verification. +- A **runtime invariant registry** where each package publishes a `./invariant` companion — and where a package with no plausible runtime relationship must ship an empty installer with a `No runtime invariant:` comment *explaining why*. Exhaustive publication, deliberately non-synthetic assertions. +- Numbered **postmortems** with a README. + +### 3.9 Smaller things codeoid doesn't have + +- **Persistent terminals** as a seam (`ctx.terminals` + `tool-terminal`), distinct from one-shot bash. +- **`jobs`** — a generic background-task runtime with owner-fenced access, `running/stopping/completed/killed/failed` status, and model-facing `job_*` controls; `bash` and `subagent` are just two registered kinds. +- **`schedule`** — durable session-local reminders (`after` / `at` / `every`, minimum five-minute interval) that return to the original live session as ordinary conversation turns. Codeoid's own COMPARISON.md lists "no scheduler yet" as a gap. +- **`skill`** — a provider registry plus filesystem implementation and a catalog/loader tool. +- **`credentials`** — a credential-*reference* seam, so config names an env var rather than carrying a secret. +- **`preset` / `persona`** — per-session agent composition from preset files, joined into every child. +- **Hook compatibility bridges** for Claude Code and Codex hook configs, with the honest framing that a native plugin would be strictly better and the bridge exists only as a compatibility path. +- **`tool-cordis`** — five model-facing tools letting the agent inspect the live plugin runtime and define/run/stop its own plugins in-process. Explicitly not a security boundary; "treat it like bash access." +- **`spill`** (§4.5). +- **`plan-mode`** as logged state. + +--- + +## 4. Ideas worth adopting, ranked + +Ordered by (payoff ÷ effort) for codeoid *as a control plane*, not as a harness. + +### 4.1 Break `session.ts` and `session-manager.ts` into capability seams — **highest leverage** + +Not Cordis. The pattern, applied by hand, exactly as codeoid already did once for `SessionProvider`. + +Take the Service Definition / Provider / Consumer triad seriously for `fs`, `shell`, `sandbox`, `compaction`, and `memory`. +The concrete prize is the one dsh names: once fs and subprocess are one seam, pointing them at a remote host or a container moves bash, PTY, and any future LSP with them — which is the natural next step for a *control plane* whose sessions need not run on the daemon's own machine. + +9,800 lines across two files is the binding constraint on everything else in this list. + +### 4.2 Collapse persistence to one append-only log with derived projections + +Adopt "model-visible means logged," classify events `current` / `shadowed` / `log-only`, and derive model history, scrollback, resume, and the UI from the one stream. + +Two immediate wins: `resume-reconcile.ts` becomes unnecessary rather than merely correct, and any future state that today would need a live mirror becomes a pure fold, the way dsh's plan mode is. + +This is the deepest change on the list and should follow 4.1. + +### 4.3 Guard plugins — cheapest real win + +`repeat-tool-reminder` and per-tool `timeoutMs` enforcement. +A few hundred lines each, no architectural prerequisites, and they target the exact failure modes of unattended dispatch and conductor runs. +Do these first while 4.1 is in flight. + +### 4.4 Runtime invariant registry + +A registry where each subsystem publishes checks over relationships tests can't see, with the "empty installer plus a written reason" rule so the publication list stays exhaustive. +Cheap, and it catches the class of bug that killed `resume-reconcile` state. + +### 4.5 Spill, backed by codeoid's recall + +Codeoid's `compress` handles commands it *recognizes*. +Spill is the catch-all for everything else: any oversized tool result → private session-scoped file, head/tail preview, opaque locator, retrieval hint. + +Codeoid can build a strictly better version than dsh's, because dsh's retrieval hint is "use read or grep on this path" while codeoid's retrieval backend is *semantic recall*. +Compress narrows what it understands; spill catches the tail; recall retrieves both. +That closes the whole surface. + +Steal the details: full content persisted verbatim, exclusive `open(path, 'wx', 0o600)` so a planted symlink cannot redirect the write, and best-effort degradation — a save failure keeps the inline result rather than turning a successful call into an error. + +### 4.6 An OS-level sandbox seam + +Codeoid's own COMPARISON.md already carries this as a gap against Omnigent. +dsh is MIT-licensed and has a working three-platform implementation, and their vocabulary is worth adopting even if the code isn't: `full | partial` enforcement reported honestly, per-backend denial dialects, `runnerFailureRules` separating runner failure from confinement working, and fail-closed with silent passthrough forbidden. + +For an identity-first product this is the missing half of the story. + +### 4.7 LSP seam + +Four operations, one stdio provider, one tool. +Bounded scope, real capability gain, and it is the natural second consumer that proves the fs/subprocess seam from 4.1 is actually a seam. + +### 4.8 Compaction rigor + +Even keeping codeoid's lossless-via-recall rotation, adopt: + +- **Tool-pairing-balanced boundaries.** Prevents a real class of provider errors. +- **A tool-result pruner as a cheap pre-summary step**, measured in code points. +- **The bracketed lock ordered so a crash leaves a detectable orphan**, not a false completion. + +### 4.9 "Model Experience" + "KV Cache effect" in every subsystem doc + +The most interesting cultural artifact in the repo, and nearly free. + +215 of 268 package READMEs state what the model sees and whether the package invalidates the prompt prefix. +For a harness, prompt-prefix stability *is* cost. +Codeoid measures cost per turn beautifully after the fact, but nothing currently forces a contributor to think about cache invalidation at design time. +A required section does. + +Pair it with **Known Limitations and Deferred Work** (220 READMEs) — honest, and it stops the same objection being re-litigated every quarter. + +### 4.10 Generated-and-verified catalogs + +`docs/architecture.yaml` exists but is not verified by CI. +Give every generator a `--check` mode and a gate. +`verify-type-equiv` — docs quoting source declarations, checked for drift — is the standout idea here and would suit codeoid's protocol package especially well. + +### 4.11 Schedule + +Durable per-session reminders delivered as ordinary conversation turns. +Already a known gap; dsh's shape (`after` / `at` / `every` with a five-minute floor, canonicalized to RFC 3339 UTC at creation) is a good starting spec. + +--- + +## 5. What not to adopt + +- **Cordis itself.** Vendored framework, large conceptual tax, and a plugin surface far wider than codeoid needs. Take the seam *pattern*; skip the runtime. +- **`typert`.** RPC/type-graph codegen solving a problem `packages/protocol` already solves at codeoid's scale. +- **The bilingual docs pipeline.** Byte-identical generated regions across two languages is impressive and irrelevant here. +- **Their own agent loop and LLM adapters.** Implementing the loop is the one thing codeoid deliberately doesn't do; doing it would forfeit the multi-backend position for a worse version of what Claude Code already ships. +- **The full process weight.** 1,386 agent notes and a six-tier CI gate matrix reflect a large team. Take 4.4, 4.9, and 4.10; leave the rest. + +--- + +## 6. On "world's best agent harness" + +Codeoid is not a harness and should not try to become one. +dsh *is* a harness — and so are Claude Code, Codex, and the Gemini CLI, all of which have more people on the loop itself than codeoid will. +Competing there means reimplementing the one layer codeoid gets for free from every backend it supports, and losing the multi-backend position to do it. + +The defensible target is **world's best control plane over harnesses**, and the three moats are already dug: identity, memory, and provider-independence. +dsh has none of the three and structurally cannot grow the first. + +What dsh has that codeoid needs is not features — it's *shape*. +A 5,287-line `session-manager.ts` is what happens when a good idea ships fast; capability seams are what makes the tenth good idea as cheap as the second. +Everything in §4 above §4.5 is really one recommendation wearing different hats: **buy the architecture, not the feature list.** + +Then the feature list gets cheap. + +--- + +*Claims verified by reading the deepseek-harness tree directly at `47f9438`, 2026-08-14. Developer preview, iterating rapidly — re-verify before relying on a dated claim.* diff --git a/src/config.ts b/src/config.ts index 906702b..c5b67d6 100644 --- a/src/config.ts +++ b/src/config.ts @@ -213,6 +213,44 @@ const CompressSchema = z minBytes: 1024, }); +/** + * Advisory guards (docs/prior-art-deepseek-harness.md §3.7). These observe the + * session and may inject model-facing advice; none of them can block a call. + * On by default — the guard is cheap, and the failure it catches (an unattended + * worker looping on one tool until its budget is gone) is expensive. + */ +const GuardSchema = z + .object({ + repeatTool: z + .object({ + enabled: z.boolean().default(true), + /** Consecutive-run lengths that trigger a reminder. Each must be >= 2. */ + thresholds: z.array(z.number().int().min(2)).nonempty().default([3, 5, 8]), + /** Tool-name patterns to track (`*` wildcard). Empty ⇒ all tools. */ + include: z.array(z.string()).default([]), + /** Tool-name patterns transparent to the chain. */ + exclude: z.array(z.string()).default(["TodoWrite", "todo_write"]), + /** Cap on arguments quoted in the reminder — never on detection. */ + argumentsPreviewChars: z.number().int().positive().default(500), + }) + .default({ + enabled: true, + thresholds: [3, 5, 8], + include: [], + exclude: ["TodoWrite", "todo_write"], + argumentsPreviewChars: 500, + }), + }) + .default({ + repeatTool: { + enabled: true, + thresholds: [3, 5, 8], + include: [], + exclude: ["TodoWrite", "todo_write"], + argumentsPreviewChars: 500, + }, + }); + const WorkspaceIndexSchema = z .object({ enabled: z.boolean().default(true), @@ -773,6 +811,7 @@ const RootSchema = z.object({ memory: MemorySchema, workspaceIndex: WorkspaceIndexSchema, compress: CompressSchema, + guard: GuardSchema, labeling: LabelingSchema, telemetry: TelemetrySchema, autoRotate: AutoRotateSchema, @@ -856,6 +895,20 @@ export interface CodeoidConfig { compressPipes: boolean; minBytes: number; }; + /** + * Advisory guards — observe and advise, never block. Optional on the type + * (like `hooks`) so a hand-built config literal need not carry it; the Zod + * schema still defaults it, so anything loaded through `loadConfig` has it. + */ + guard?: { + repeatTool: { + enabled: boolean; + thresholds: number[]; + include: string[]; + exclude: string[]; + argumentsPreviewChars: number; + }; + }; /** Cluster-label settings (Haiku API key). */ labeling: { anthropicApiKey?: string; @@ -1059,6 +1112,8 @@ const ENV_OVERRIDES: readonly EnvOverride[] = [ { env: "CODEOID_COMPRESS_EXCLUDE_PATTERNS", path: "compress.excludePatterns", kind: "csv" }, { env: "CODEOID_COMPRESS_PIPES", path: "compress.compressPipes", kind: "boolean" }, { env: "CODEOID_COMPRESS_MIN_BYTES", path: "compress.minBytes", kind: "int" }, + { env: "CODEOID_GUARD_REPEAT_TOOL", path: "guard.repeatTool.enabled", kind: "boolean" }, + { env: "CODEOID_GUARD_REPEAT_TOOL_EXCLUDE", path: "guard.repeatTool.exclude", kind: "csv" }, { env: "ANTHROPIC_API_KEY", path: "labeling.anthropicApiKey", kind: "string" }, { env: "CODEOID_OSC8", path: "telemetry.osc8", kind: "string" }, { env: "CODEOID_AUTO_ROTATE", path: "autoRotate.enabled", kind: "boolean" }, @@ -1279,6 +1334,7 @@ export function loadConfig(opts: LoadOptions = {}): CodeoidConfig { }, workspaceIndex: parsed.workspaceIndex, compress: parsed.compress, + guard: parsed.guard, labeling: parsed.labeling, telemetry: { osc8: osc8Mode }, autoRotate: parsed.autoRotate, diff --git a/src/daemon/guard/index.ts b/src/daemon/guard/index.ts new file mode 100644 index 0000000..66cf612 --- /dev/null +++ b/src/daemon/guard/index.ts @@ -0,0 +1,21 @@ +/** + * Guard — advisory plugins that improve agent hygiene without taking authority. + * + * A guard observes the session's event stream and may inject model-facing + * advice. It never vetoes a tool call, never rewrites arguments, and never + * appears in the tool list. Anything that needs to *stop* a call belongs in the + * approval flow or the autonomous budget, not here. + * + * See docs/prior-art-deepseek-harness.md §3.7. + */ + +export { + RepeatToolGuard, + DEFAULT_REPEAT_TOOL_CONFIG, + PRIMARY_CHAIN, + canonicalizeArguments, + matchesToolPattern, + normalizeRepeatToolConfig, + type RepeatToolGuardConfig, + type RepeatToolReminder, +} from "./repeat-tool.js"; diff --git a/src/daemon/guard/repeat-tool.test.ts b/src/daemon/guard/repeat-tool.test.ts new file mode 100644 index 0000000..65c35ca --- /dev/null +++ b/src/daemon/guard/repeat-tool.test.ts @@ -0,0 +1,228 @@ +import { describe, test, expect } from "bun:test"; +import { + RepeatToolGuard, + PRIMARY_CHAIN, + canonicalizeArguments, + matchesToolPattern, + normalizeRepeatToolConfig, +} from "./repeat-tool"; + +const read = (p: string) => ({ file_path: p }); + +describe("canonicalizeArguments", () => { + test("key order does not change the canonical form", () => { + expect(canonicalizeArguments({ a: 1, b: 2 })).toBe( + canonicalizeArguments({ b: 2, a: 1 }), + ); + }); + + test("nested key order is also normalized", () => { + expect(canonicalizeArguments({ o: { x: 1, y: 2 }, a: 3 })).toBe( + canonicalizeArguments({ a: 3, o: { y: 2, x: 1 } }), + ); + }); + + test("array order is preserved — it is meaningful", () => { + expect(canonicalizeArguments({ a: [1, 2] })).not.toBe( + canonicalizeArguments({ a: [2, 1] }), + ); + }); + + test("differing values stay distinct", () => { + expect(canonicalizeArguments(read("a.ts"))).not.toBe( + canonicalizeArguments(read("b.ts")), + ); + }); + + test("cycles degrade instead of throwing", () => { + const cyclic: Record = { a: 1 }; + cyclic.self = cyclic; + expect(canonicalizeArguments(cyclic)).toContain("[circular]"); + }); + + test("undefined is not trackable", () => { + expect(canonicalizeArguments(undefined)).toBeNull(); + }); +}); + +describe("matchesToolPattern", () => { + test("exact match, case-insensitive across backend spellings", () => { + expect(matchesToolPattern("TodoWrite", "todowrite")).toBe(true); + expect(matchesToolPattern("todo_write", "TODO_WRITE")).toBe(true); + }); + + test("wildcard", () => { + expect(matchesToolPattern("mcp__memory__recall", "mcp__*")).toBe(true); + expect(matchesToolPattern("Read", "mcp__*")).toBe(false); + }); + + test("regex metacharacters in a pattern are literal", () => { + expect(matchesToolPattern("a.b", "a.b")).toBe(true); + expect(matchesToolPattern("axb", "a.b")).toBe(false); + }); +}); + +describe("normalizeRepeatToolConfig", () => { + test("sorts thresholds ascending", () => { + expect(normalizeRepeatToolConfig({ thresholds: [8, 3, 5] }).thresholds).toEqual([3, 5, 8]); + }); + + test("fails loud rather than silently defaulting", () => { + expect(() => normalizeRepeatToolConfig({ thresholds: [] })).toThrow(/must not be empty/); + expect(() => normalizeRepeatToolConfig({ thresholds: [1] })).toThrow(/>= 2/); + expect(() => normalizeRepeatToolConfig({ thresholds: [3, 3] })).toThrow(/duplicates/); + expect(() => normalizeRepeatToolConfig({ thresholds: [2.5] })).toThrow(/integers/); + expect(() => normalizeRepeatToolConfig({ argumentsPreviewChars: 0 })).toThrow(/>= 1/); + }); +}); + +describe("RepeatToolGuard", () => { + test("fires at each configured threshold and nowhere else", () => { + const g = new RepeatToolGuard({ thresholds: [3, 5] }); + const fired: number[] = []; + for (let i = 0; i < 7; i++) { + const r = g.observe(PRIMARY_CHAIN, "Read", read("a.ts")); + if (r) fired.push(r.runLength); + } + expect(fired).toEqual([3, 5]); + }); + + test("first threshold is the brief nudge, later ones are detailed", () => { + const g = new RepeatToolGuard({ thresholds: [3, 5] }); + let brief: boolean | undefined; + let detailed: boolean | undefined; + for (let i = 0; i < 5; i++) { + const r = g.observe(PRIMARY_CHAIN, "Read", read("a.ts")); + if (r?.runLength === 3) brief = r.brief; + if (r?.runLength === 5) detailed = r.brief; + } + expect(brief).toBe(true); + expect(detailed).toBe(false); + }); + + test("the detailed reminder quotes the arguments; the brief one does not", () => { + const g = new RepeatToolGuard({ thresholds: [2, 3] }); + g.observe(PRIMARY_CHAIN, "Read", read("secret-path.ts")); + const first = g.observe(PRIMARY_CHAIN, "Read", read("secret-path.ts")); + const second = g.observe(PRIMARY_CHAIN, "Read", read("secret-path.ts")); + expect(first?.text).not.toContain("secret-path.ts"); + expect(second?.text).toContain("secret-path.ts"); + }); + + test("a different argument resets the run", () => { + const g = new RepeatToolGuard({ thresholds: [3] }); + g.observe(PRIMARY_CHAIN, "Read", read("a.ts")); + g.observe(PRIMARY_CHAIN, "Read", read("a.ts")); + g.observe(PRIMARY_CHAIN, "Read", read("b.ts")); // resets to 1 + expect(g.observe(PRIMARY_CHAIN, "Read", read("b.ts"))).toBeNull(); + expect(g.runLength(PRIMARY_CHAIN)).toBe(2); + }); + + test("a different tool resets the run", () => { + const g = new RepeatToolGuard({ thresholds: [3] }); + g.observe(PRIMARY_CHAIN, "Read", read("a.ts")); + g.observe(PRIMARY_CHAIN, "Read", read("a.ts")); + g.observe(PRIMARY_CHAIN, "Grep", { pattern: "x" }); + expect(g.observe(PRIMARY_CHAIN, "Read", read("a.ts"))).toBeNull(); + }); + + test("key order in the input does not break a run", () => { + const g = new RepeatToolGuard({ thresholds: [3] }); + g.observe(PRIMARY_CHAIN, "Edit", { a: 1, b: 2 }); + g.observe(PRIMARY_CHAIN, "Edit", { b: 2, a: 1 }); + expect(g.observe(PRIMARY_CHAIN, "Edit", { a: 1, b: 2 })?.runLength).toBe(3); + }); + + test("excluded tools are fully transparent — they neither advance nor reset", () => { + const g = new RepeatToolGuard({ thresholds: [3], exclude: ["TodoWrite"] }); + g.observe(PRIMARY_CHAIN, "Read", read("a.ts")); + g.observe(PRIMARY_CHAIN, "Read", read("a.ts")); + expect(g.observe(PRIMARY_CHAIN, "TodoWrite", { todos: [] })).toBeNull(); + expect(g.observe(PRIMARY_CHAIN, "Read", read("a.ts"))?.runLength).toBe(3); + }); + + test("include narrows tracking to the listed tools", () => { + const g = new RepeatToolGuard({ thresholds: [2], include: ["Bash"] }); + expect(g.observe(PRIMARY_CHAIN, "Read", read("a.ts"))).toBeNull(); + expect(g.observe(PRIMARY_CHAIN, "Read", read("a.ts"))).toBeNull(); + g.observe(PRIMARY_CHAIN, "Bash", { command: "ls" }); + expect(g.observe(PRIMARY_CHAIN, "Bash", { command: "ls" })?.runLength).toBe(2); + }); + + test("a pattern matching no live tool is not an error", () => { + const g = new RepeatToolGuard({ thresholds: [2], exclude: ["mcp__*"] }); + expect(g.observe(PRIMARY_CHAIN, "Read", read("a.ts"))).toBeNull(); + expect(g.observe(PRIMARY_CHAIN, "Read", read("a.ts"))?.runLength).toBe(2); + }); + + test("chains are independent per agent — parallel subagents do not interleave", () => { + const g = new RepeatToolGuard({ thresholds: [3] }); + for (let i = 0; i < 2; i++) { + g.observe("agent-a", "Read", read("a.ts")); + g.observe("agent-b", "Read", read("b.ts")); + } + // Interleaved calls would have reset a single shared chain to 1 each time. + expect(g.observe("agent-a", "Read", read("a.ts"))?.runLength).toBe(3); + expect(g.observe("agent-b", "Read", read("b.ts"))?.runLength).toBe(3); + }); + + test("the argument preview is capped but detection uses the full string", () => { + const g = new RepeatToolGuard({ thresholds: [2, 3], argumentsPreviewChars: 20 }); + const big = { content: "x".repeat(5000) }; + g.observe(PRIMARY_CHAIN, "Write", big); + g.observe(PRIMARY_CHAIN, "Write", big); + const r = g.observe(PRIMARY_CHAIN, "Write", big); + expect(r?.runLength).toBe(3); // full-string comparison still matched + expect(r!.text).toContain("more characters omitted"); + expect(r!.text.length).toBeLessThan(1000); // preview bounded the text + }); + + test("two large payloads differing only past the preview cap are distinct", () => { + const g = new RepeatToolGuard({ thresholds: [2], argumentsPreviewChars: 10 }); + g.observe(PRIMARY_CHAIN, "Write", { content: `${"x".repeat(500)}A` }); + expect(g.observe(PRIMARY_CHAIN, "Write", { content: `${"x".repeat(500)}B` })).toBeNull(); + }); + + test("the tool/arguments key separator cannot be forged from a tool name", () => { + // The chain key joins tool name and canonical args. With an ordinary + // separator (a space), a tool literally named `Read {"x":1}` calling with + // `{}` would key the same as `Read` calling with `{"x":1}`. NUL can't + // appear in a tool name, and JSON.stringify escapes it, so it can't + // appear in the canonical args either. + const g = new RepeatToolGuard({ thresholds: [2] }); + g.observe(PRIMARY_CHAIN, 'Read {"x":1}', {}); + expect(g.observe(PRIMARY_CHAIN, "Read", { x: 1 })).toBeNull(); + expect(g.runLength(PRIMARY_CHAIN)).toBe(1); + }); + + test("resetChain drops a run — an owner redirect invalidates it", () => { + const g = new RepeatToolGuard({ thresholds: [3] }); + g.observe(PRIMARY_CHAIN, "Read", read("a.ts")); + g.observe(PRIMARY_CHAIN, "Read", read("a.ts")); + g.resetChain(PRIMARY_CHAIN); + expect(g.observe(PRIMARY_CHAIN, "Read", read("a.ts"))).toBeNull(); + expect(g.runLength(PRIMARY_CHAIN)).toBe(1); + }); + + test("disabled guard never fires", () => { + const g = new RepeatToolGuard({ enabled: false, thresholds: [2] }); + g.observe(PRIMARY_CHAIN, "Read", read("a.ts")); + expect(g.observe(PRIMARY_CHAIN, "Read", read("a.ts"))).toBeNull(); + }); + + test("unserialisable input drops the chain rather than guessing", () => { + const g = new RepeatToolGuard({ thresholds: [2] }); + g.observe(PRIMARY_CHAIN, "Read", read("a.ts")); + expect(g.observe(PRIMARY_CHAIN, "Read", undefined)).toBeNull(); + expect(g.runLength(PRIMARY_CHAIN)).toBe(0); + }); + + test("the advisory is marked as daemon-authored, not owner input", () => { + const g = new RepeatToolGuard({ thresholds: [2] }); + g.observe(PRIMARY_CHAIN, "Read", read("a.ts")); + const r = g.observe(PRIMARY_CHAIN, "Read", read("a.ts")); + expect(r!.text).toContain("NOT a message from the owner"); + expect(r!.text).toContain(""); + expect(r!.text).toContain(""); + }); +}); diff --git a/src/daemon/guard/repeat-tool.ts b/src/daemon/guard/repeat-tool.ts new file mode 100644 index 0000000..71d96d8 --- /dev/null +++ b/src/daemon/guard/repeat-tool.ts @@ -0,0 +1,312 @@ +/** + * Repeat-tool guard — an advisory loop-breaker. + * + * Prior art: DeepSeek Harness `dsh-repeat-tool-reminder` + * (docs/prior-art-deepseek-harness.md §3.7 / §4.3). Their framing is the right + * one and is preserved here: this is NOT a model-facing tool. It never appears + * in the tool list, never vetoes a call, never rewrites arguments, and never + * delays anything. It adds exactly one behaviour — it watches each agent's + * stream of tool calls, counts runs of consecutive calls to the same tool with + * identical canonicalised arguments, and at configured run lengths injects an + * escalating advisory telling the model to stop repeating itself, re-read the + * last result, and either change approach or conclude. + * + * The decision (retry differently, gather more evidence, or finish) stays + * entirely with the model. A legitimately repeated call is blocked by nothing. + * + * Why codeoid wants it: unattended work — `dispatch` workers and conductor- + * spawned sessions — has no human watching the transcript. A model wedged on + * `Read(same file)` or `Bash(same failing command)` burns the tool budget to + * zero and reports failure with no diagnosis. The budget caps the damage; this + * catches the cause while the turn can still recover. + * + * Two deliberate adaptations from dsh's version: + * + * - **Chains are keyed per emitting agent**, because codeoid's `tool_start` + * carries `sdkAgentId` for subagent calls. Two subagents hammering the same + * tool in parallel are two independent chains, not one interleaved mess that + * resets constantly and never fires. + * - **Pattern matching is case-insensitive**, because codeoid is multi-backend + * and the same logical tool is `TodoWrite` on Claude and `todo_write` + * elsewhere. A single default exclude list has to cover both spellings. + * + * This module is pure: no I/O, no clock, no daemon imports. All state is the + * per-chain counters held in the instance. + */ + +/** Tuning for the repeat-tool guard. See `DEFAULT_REPEAT_TOOL_CONFIG`. */ +export interface RepeatToolGuardConfig { + /** Master switch. */ + enabled: boolean; + /** + * Consecutive-run lengths that trigger a reminder, e.g. `[3, 5, 8]`. The + * FIRST threshold delivers a short generic nudge; every later threshold + * delivers the detailed form naming the tool, the run length, and the + * canonical arguments. + */ + thresholds: readonly number[]; + /** Tool-name patterns to track (`*` wildcard). Empty ⇒ track everything. */ + include: readonly string[]; + /** Tool-name patterns transparent to the chain (`*` wildcard). */ + exclude: readonly string[]; + /** + * Cap on the arguments quoted in the detailed reminder. Bounds only the + * reminder text — the chain key always compares the FULL canonical string, + * so a looping `Write`/`Edit` payload can neither defeat detection nor ride + * into the next request unbounded. + */ + argumentsPreviewChars: number; +} + +export const DEFAULT_REPEAT_TOOL_CONFIG: RepeatToolGuardConfig = { + enabled: true, + thresholds: [3, 5, 8], + include: [], + // Todo bookkeeping is legitimately called repeatedly with near-identical + // input and is never the tool a session is wedged on. Both spellings, since + // backends disagree. + exclude: ["TodoWrite", "todo_write"], + argumentsPreviewChars: 500, +}; + +/** One advisory the guard wants injected before the model's next request. */ +export interface RepeatToolReminder { + /** Tool the chain is stuck on. */ + toolName: string; + /** Consecutive identical calls observed, including the one that fired this. */ + runLength: number; + /** Which configured threshold fired. */ + threshold: number; + /** True for the first threshold — the short generic nudge. */ + brief: boolean; + /** Model-facing advisory text, ready to inject. */ + text: string; +} + +/** The primary agent's chain id, used when a `tool_start` carries no agent. */ +export const PRIMARY_CHAIN = "primary"; + +/** + * Deep key-sort + `JSON.stringify`, so argument objects differing only in + * property order count as identical. Arrays keep their order (order is + * meaningful in a tool argument list); only object keys are sorted. + * + * Tool inputs originate as JSON so cycles are not expected, but a provider + * handing us a live object graph must not crash the event consumer — an + * unserialisable input yields `null`, which the caller treats as "not + * trackable" rather than as a chain key that might collide. + */ +export function canonicalizeArguments(input: unknown): string | null { + const sortDeep = (value: unknown, seen: Set): unknown => { + if (value === null || typeof value !== "object") return value; + if (seen.has(value as object)) return "[circular]"; + seen.add(value as object); + try { + if (Array.isArray(value)) return value.map((v) => sortDeep(v, seen)); + const out: Record = {}; + for (const key of Object.keys(value as Record).sort()) { + out[key] = sortDeep((value as Record)[key], seen); + } + return out; + } finally { + seen.delete(value as object); + } + }; + try { + const canonical = JSON.stringify(sortDeep(input, new Set())); + // `undefined` input stringifies to undefined, not a string. + return canonical ?? null; + } catch { + return null; + } +} + +/** + * Case-insensitive glob match supporting `*` only. Patterns are predicates + * over whatever tools exist at call time, NOT references to a registry — a + * pattern matching no currently registered tool is not an error, so + * `exclude: ["mcp__*"]` stays valid in a session that mounts no MCP servers. + */ +export function matchesToolPattern(toolName: string, pattern: string): boolean { + const escaped = pattern.replace(/[.+?^${}()|[\]\\]/g, "\\$&").replace(/\*/g, ".*"); + return new RegExp(`^${escaped}$`, "i").test(toolName); +} + +/** + * Validate and normalise config. Fails loud rather than silently falling back + * to defaults — a typo'd threshold list is a configuration bug, and silently + * ignoring it means the guard never fires and nobody finds out until a worker + * has burned its budget in a loop. + * + * @throws when thresholds are empty, non-integer, below 2, or duplicated, or + * when `argumentsPreviewChars` is not a positive integer. + */ +export function normalizeRepeatToolConfig( + config: Partial = {}, +): RepeatToolGuardConfig { + const merged = { ...DEFAULT_REPEAT_TOOL_CONFIG, ...config }; + const thresholds = [...merged.thresholds]; + + if (thresholds.length === 0) { + throw new Error("guard.repeatTool.thresholds must not be empty"); + } + for (const t of thresholds) { + if (!Number.isInteger(t)) { + throw new Error(`guard.repeatTool.thresholds must be integers (got ${t})`); + } + // A threshold of 1 would fire on every first call — that is not a repeat. + if (t < 2) { + throw new Error(`guard.repeatTool.thresholds must be >= 2 (got ${t})`); + } + } + if (new Set(thresholds).size !== thresholds.length) { + throw new Error(`guard.repeatTool.thresholds must not contain duplicates (got ${thresholds.join(", ")})`); + } + if ( + !Number.isInteger(merged.argumentsPreviewChars) || + merged.argumentsPreviewChars < 1 + ) { + throw new Error( + `guard.repeatTool.argumentsPreviewChars must be an integer >= 1 (got ${merged.argumentsPreviewChars})`, + ); + } + + thresholds.sort((a, b) => a - b); + return { ...merged, thresholds }; +} + +/** Per-chain state: what the last tracked call was, and how many in a row. */ +interface Chain { + key: string; + toolName: string; + canonicalArgs: string; + count: number; +} + +/** + * Tracks consecutive-identical tool calls per agent and yields reminders. + * + * Lifecycle: one instance per Session. `observe()` on every `tool_start`, + * `resetChain()` when real owner input arrives (a redirect invalidates the + * run), and `forget()` when a subagent stops so its chain doesn't leak. + */ +export class RepeatToolGuard { + readonly config: RepeatToolGuardConfig; + readonly #chains = new Map(); + + constructor(config: Partial = {}) { + this.config = normalizeRepeatToolConfig(config); + } + + /** Is this tool tracked at all? `include` empty ⇒ everything but `exclude`. */ + #tracks(toolName: string): boolean { + if (this.config.exclude.some((p) => matchesToolPattern(toolName, p))) return false; + if (this.config.include.length === 0) return true; + return this.config.include.some((p) => matchesToolPattern(toolName, p)); + } + + /** + * Record one tool call and return a reminder when the run length just hit a + * configured threshold, else `null`. + * + * An untracked tool is fully transparent: it neither advances nor resets a + * chain, so `Read, Read, TodoWrite, Read` is a run of three Reads. + * + * @param chainId emitting agent — `sdkAgentId`, or `PRIMARY_CHAIN`. + * @param toolName the tool being called. + * @param input raw tool input; canonicalised internally. + */ + observe( + chainId: string, + toolName: string, + input: unknown, + ): RepeatToolReminder | null { + if (!this.config.enabled) return null; + if (!this.#tracks(toolName)) return null; + + const canonicalArgs = canonicalizeArguments(input); + // Unserialisable input can't be compared, so it can't prove a repeat. + // Drop the chain rather than guess — a false reminder is worse than none. + if (canonicalArgs === null) { + this.#chains.delete(chainId); + return null; + } + + const key = `${toolName}\u0000${canonicalArgs}`; + const prev = this.#chains.get(chainId); + const count = prev && prev.key === key ? prev.count + 1 : 1; + this.#chains.set(chainId, { key, toolName, canonicalArgs, count }); + + if (!this.config.thresholds.includes(count)) return null; + + const brief = count === this.config.thresholds[0]; + return { + toolName, + runLength: count, + threshold: count, + brief, + text: this.#render(toolName, canonicalArgs, count, brief), + }; + } + + /** + * Drop one chain. Called when the owner sends real input (the redirect makes + * the prior run stale) and when a subagent stops. + */ + resetChain(chainId: string): void { + this.#chains.delete(chainId); + } + + /** Drop every chain — session rotation, backend switch, fork. */ + resetAll(): void { + this.#chains.clear(); + } + + /** Current run length for a chain. Testing and diagnostics. */ + runLength(chainId: string): number { + return this.#chains.get(chainId)?.count ?? 0; + } + + /** + * The advisory itself. Wrapped in a tagged block and explicitly marked as + * daemon-authored, matching the `` convention in + * `Session#maybeDeliverBackgroundReports` — the model must never mistake an + * injected advisory for a message from the owner. + */ + #render( + toolName: string, + canonicalArgs: string, + count: number, + brief: boolean, + ): string { + const head = [ + "", + "(daemon-injected advisory — NOT a message from the owner)", + ]; + if (brief) { + return [ + ...head, + `You have called \`${toolName}\` ${count} times in a row with identical arguments.`, + "Re-read the result you already have. If it answered the question, move on; if it did not, change your approach rather than repeating the call.", + "", + ].join("\n"); + } + const cap = this.config.argumentsPreviewChars; + const preview = + canonicalArgs.length > cap + ? `${canonicalArgs.slice(0, cap)}… (${canonicalArgs.length - cap} more characters omitted)` + : canonicalArgs; + return [ + ...head, + `You have now called \`${toolName}\` ${count} times consecutively with identical arguments:`, + "", + preview, + "", + "This is a loop. The result will not change. Do one of these instead:", + " 1. Re-read the last result — the answer is very likely already in it.", + " 2. Change the arguments, or use a different tool, to get new information.", + " 3. Stop and report what you know, including that this step is blocked and why.", + "", + ].join("\n"); + } +} diff --git a/src/daemon/session.ts b/src/daemon/session.ts index 1b2663b..8b1dd6d 100644 --- a/src/daemon/session.ts +++ b/src/daemon/session.ts @@ -83,6 +83,7 @@ import type { Attachment } from "../protocol/types.js"; import { resolveAttachments } from "./attachments.js"; import type { CodeoidConfig } from "../config.js"; import type { CompressionRegistry } from "./compress/index.js"; +import { PRIMARY_CHAIN, RepeatToolGuard } from "./guard/index.js"; import { CLAUDE_PROVIDER_ID, findModel, @@ -373,6 +374,13 @@ export class Session { #compressionRegistry?: CompressionRegistry; #onModels?: SessionCreateOptions["onModels"]; #hookBus?: HookBus; + /** + * Advisory loop-breaker. Watches `tool_start` for runs of consecutive + * identical calls and injects a reminder — never blocks. Undefined when the + * guard is disabled, or when its config failed validation (the guard is + * optional, so a bad threshold list must not take the session down). + */ + #repeatGuard?: RepeatToolGuard; #status: SessionStatus = "idle"; /** True from an interrupt() until the next turn STARTS. An interrupt leaves @@ -702,6 +710,19 @@ export class Session { this.#compressionRegistry = opts.compressionRegistry; this.#onModels = opts.onModels; this.#hookBus = opts.hooks; + // Advisory guard. Config is validated in the guard constructor and fails + // loud there; here we degrade to "no guard" and log, because an advisory + // plugin must never be the reason a session refuses to start. + const repeatCfg = opts.config?.guard?.repeatTool; + if (repeatCfg?.enabled !== false) { + try { + this.#repeatGuard = new RepeatToolGuard(repeatCfg); + } catch (err) { + console.error( + `[codeoid/session ${this.id.slice(0, 8)}] repeat-tool guard disabled — invalid config: ${err instanceof Error ? err.message : String(err)}`, + ); + } + } // Tenant-scoped (auth carries account_id/project_id) so two accounts in // the same directory never share memory. this.#workspaceId = workspaceIdFromPath(opts.workdir, opts.auth); @@ -1628,6 +1649,11 @@ export class Session { this.#store.audit(sender.sub, "session.send", this.id); // A new turn is starting — clear any interrupt from the previous one. this.#turnInterrupted = false; + // Any inbound message invalidates a repeat run: the guard only claims + // "N identical calls with nothing else happening", and this is something + // else happening. Applies to system principals too (a background-task + // digest or a dispatch task is new information by the same argument). + this.#repeatGuard?.resetAll(); // Fork-setup gate: on a freshly-forked worktree the first turn must wait // for `fork.setup` (e.g. `bun install`) to finish, so the agent never @@ -1944,6 +1970,65 @@ export class Session { } } + /** + * Feed one tool call to the repeat-tool guard and inject its advisory when a + * run of consecutive identical calls hits a configured threshold. + * + * Advisory only: nothing here blocks, denies, or delays the call that + * triggered it — that call has already been recorded and is on its way to + * approval or execution. The reminder lands in the model's NEXT request. + * + * Injection uses `later` priority, which merges into the running turn + * without starting a fresh query (see the priority notes in `#sendInner`). + * That is precisely "add context to the next request" and is why the guard + * costs no extra turn. A backend without `pushMidTurn` silently gets no + * reminders rather than a queued message that would arrive out of context. + */ + #maybeInjectRepeatReminder( + toolName: string, + input: Record, + sdkAgentId?: string, + ): void { + const guard = this.#repeatGuard; + if (!guard) return; + + const reminder = guard.observe(sdkAgentId ?? PRIMARY_CHAIN, toolName, input); + if (!reminder) return; + + const run = this.#activeRun; + if (!run?.pushMidTurn) return; + + // Operator-visible trace. Not sent to the model — the advisory itself is. + const infoMsg = this.#makeMessage( + "info", + `⟳ Repeat-tool guard — ${toolName} called ${reminder.runLength}× with identical arguments; advisory injected`, + SYSTEM_IDENTITY, + undefined, + undefined, + { + event: "repeat_tool_reminder", + tool: toolName, + runLength: reminder.runLength, + agentId: sdkAgentId ?? PRIMARY_CHAIN, + }, + ); + this.#persistAndBuffer(infoMsg); + this.#broadcastRaw(infoMsg); + + // Model-visible means logged: the advisory reaches the model, so canonical + // history has to carry it or a cross-backend fork would replay a + // conversation the model never actually had. + this.#accumulator.pushUserTurn(reminder.text); + run.pushMidTurn(reminder.text, "later"); + + this.#store.audit( + "system:guard", + "session.repeat_tool_reminder", + this.id, + `tool=${toolName} run=${reminder.runLength} agent=${sdkAgentId ?? PRIMARY_CHAIN}`, + ); + } + /** * Deliver queued settle digests as ONE injected wake turn, iff the session * is idle. Called from two triggers so both arrival orders work: when a @@ -3837,6 +3922,7 @@ export class Session { } this.#persistAndBuffer(toolMsg); this.#broadcastRaw(toolMsg); + this.#maybeInjectRepeatReminder(event.name, event.input, event.sdkAgentId); if (!autoApprove) this.#setStatus("waiting_approval"); break; } @@ -3926,6 +4012,9 @@ export class Session { const agentId = event.agentId; void this.#identityManager?.deactivateSubagent(this.id, agentId); this.#subagentRegistrations.delete(agentId); + // Drop the guard chain with the agent, or a long session accumulates + // one dead chain per subagent it ever spawned. + this.#repeatGuard?.resetChain(agentId); if (this.#subagents.delete(agentId)) { this.#broadcastInfoUpdate(); }