diff --git a/aidd_docs/memory/backlog.md b/aidd_docs/memory/backlog.md index 7b30f72c3..a502849b3 100644 --- a/aidd_docs/memory/backlog.md +++ b/aidd_docs/memory/backlog.md @@ -8,7 +8,7 @@ | AIDD Roadmap board (org project 8) | order, status, and what ships next | the single source of truth for the roadmap | | Milestones | themes | one milestone per theme; no milestone means backlog | | GitHub Discussions | ideas and their upvotes | where an idea is weighed before it becomes an issue | -| `aidd_docs/` task documents | the local plans, specs and reviews behind an issue | never a substitute for the issue | +| `aidd_docs/` task documents | the local contract, plan and evidence behind an issue | never a substitute for the issue | ## Representation @@ -43,4 +43,5 @@ The board is moved by hand, by a human or an agent through `gh`. No workflow adv ## Relations - An issue's branch and pull request carry the routing: the branch prefix decides the target, per `vcs.md`. -- A `aidd_docs/` task folder is grouped by its parent document (`plan.md`, then `master-plan.md`, then `spec.md`), and a sub-document's status never moves that parent. +- A new `aidd_docs/` task folder holds `contract.md`, `plan.md` with its phases folded in, and `evidence.md`. The review lives in the pull request, not in a file, and no executable code or prototype goes in the folder: probes stay in the scratchpad. In this repository this overrides the `phase-.md` that `aidd-dev:01-plan` writes and `aidd-dev:02-implement` reads, and the `review.md` that `aidd-dev:05-review` writes; folders made before this rule keep their files. +- A task folder is grouped by its parent document (`plan.md`, then `master-plan.md`, then `spec.md`), which goes to a Done status (`done` or `implemented`) when its pull request merges. A sub-document's status never moves that parent. diff --git a/aidd_docs/product/usage-contract.md b/aidd_docs/product/usage-contract.md index 48e432e47..e097f6561 100644 --- a/aidd_docs/product/usage-contract.md +++ b/aidd_docs/product/usage-contract.md @@ -49,7 +49,7 @@ The ledger stores `StoredUsage`: the record plus `repository_id`, resolved at in **Null is not zero.** - Unknown is `null`, never `0`. A `null` counter is left out of its sum and counted in that counter's `unknown_records`. - A usage line missing a counter is still a record, that counter `null`. -- A line that is no usage line (not JSON, no `message.usage`, no `message.id`, `sessionId` or `timestamp`) is never a record. It counts in `run.unrecognised_shapes` of the report that read it, and, when it names a clone that consents, in `coverage.left_out.unrecognised` of every report (see [Left out](#left-out)). +- A line of a shape the reader does not recognise is never a record: one that is not JSON or not an object, or an `assistant` line whose `message.usage` is not an object or that lacks `message.id`, `sessionId` or `timestamp`. It counts in `run.unrecognised_shapes` of the report that read it, and, when it names a clone that consents, in `coverage.left_out.unrecognised` of every report (see [Left out](#left-out)). A line that is not `assistant`, has no `message.usage` or names the model `` is no call at all, and counts nowhere. - Ingest and report say: "not recognised by this read: a call missing a counter is kept with that counter unknown; a line that is no usage line is never a call." (`UNRECOGNISED_WORDS`, `presentation/display/telemetry-display.ts`). What is kept of such lines is said apart: "Kept from every read: …". **Key and fold** (`domain/formats/claude-transcript-usage.ts`). @@ -98,33 +98,13 @@ What an ingest measured and could not store is kept in `ledger/left-out.txt`, so ## Report envelope -`aidd telemetry report --json` prints one object, `version: 2`. What a figure is, what it depends on, what `run` holds, and how the envelope may change are defined once, in [the envelope reference](../../plugins/aidd-telemetry/skills/01-usage/references/envelope.md#figures-and-run). Type: `ReportEnvelope`, `cli/src/contexts/telemetry/application/report/report-envelope.ts`. +`aidd telemetry report --json` prints one object, `version: 2`. Every field, what a figure is, what it depends on, what `run` holds, and how the envelope may change are defined once, in [the envelope reference](../../plugins/aidd-telemetry/skills/01-usage/references/envelope.md). Type: `ReportEnvelope`, `cli/src/contexts/telemetry/application/report/report-envelope.ts`. What the reference does not say: -| Field | Meaning | -| --- | --- | -| `version` | `2` | -| `period` | `{from, to}`, UTC days, `null` for open, as resolved: `--days` against today's UTC day. The figures depend on these days, not on the flags | -| `axis` | `total`, `person`, `session`, `model`, `agent`, `day`, `repository`, `task`, `ticket` | -| `filters` | `task`, `ticket`, `person`, `model`, `agent`, `session`, `repository`, `absent`, `unattributed`, in that order: `null` when not given, else as resolved (see [Filters](#filters)) | -| `rows` | one per axis value, of the calls that match | -| `totals` | the calls of the period that match | -| `coverage` | `records`, `left_out`. `records` counts the calls of the period, matching or not; `left_out` (`undated`, `unrecognised`) what every ingest so far measured and could not store, see [Left out](#left-out) | -| `run` | last: `files_read`, `unrecognised_shapes`, `not_stored`, `consent_log_damaged`, `oldest_transcript_at`, `skipped_ledger_lines`, what the read that printed the report found | - -Row and `totals`: `records`, `input`, `output`, `cache_read`, `cache_write` (each `{tokens, unknown_records}`), `total`. A row also has: - -| Field | Values | -| --- | --- | -| `kind` | `value`, `unattributed`, `absent` | -| `key` | axis value for `value`, else `null` | -| `reason` | `no-binding` or `declared-none` for `unattributed`, else `null` | - -- Every axis partitions the same calls: each adds up to `totals`, counter by counter. - Envelope version 1 put the figures and the run together. Version 2 keeps them apart and has no compatibility shape. - `AIDD_TELEMETRY=0`: nothing read, object is `{"version": 2, "refused": "AIDD_TELEMETRY=0"}`. - The text report labels an `absent` person `anonymous`; the envelope keeps `kind: absent`, `key: null`. -- A report ingests first; what that read found is its `run`, see [the envelope reference](../../plugins/aidd-telemetry/skills/01-usage/references/envelope.md#run). `coverage.records` describes the period; `coverage.left_out` every ingest so far. -- A report with `--days`, `--from` or `--to` reads only the days of its period and the day on each side, plus any monthly file one of them falls in, with every day of its month; with none it reads the whole ledger. The copies of a call with a `requestId` have been observed less than a day apart, and under that bound both print the same figures; a call with none is held once by the ingest that stores it, so no bound is needed. Outside these, only a report with no period folds every copy of a call: copies an older CLI stores after the migration are one such case. `run.skipped_ledger_lines` counts the lines of the partitions read. +- A report ingests first: what that read found is its `run`. +- A report with `--days`, `--from` or `--to` reads only the days of its period and the day on each side; with none it reads the whole ledger. The copies of a call with a `requestId` have been observed less than a day apart, and under that bound both print the same figures; a call with none is held once by the ingest that stores it, so no bound is needed. Outside these, only a report with no period folds every copy of a call: copies an older CLI stores after the migration are one such case. - No path, working directory or branch. `session` keys are full session ids, `repository` keys are hashes. ### Filters @@ -161,7 +141,7 @@ Directory: `AIDD_TELEMETRY_DIR`, else `AIDD_USER_CONFIG_DIR/telemetry`, else `XD | Store | Writer | Readers | Holds | | --- | --- | --- | --- | -| `ledger/.jsonl` | `ingest`, `report` | `report`, `forget` | `StoredUsage` by UTC day of `at`, in time then key order, one call per line. Ingest reads its calls' days and the day on each side (every day, and so every monthly file, when one has no `requestId`), and rewrites only a changed day, whole, by rename; no new bytes, no write. `report` rewrites, folded, the partitions it read that share a call or hold a line that is not a record; a report with no period reads every partition, so after it each call is on disk once. A line that is not a record (a newer version's, a hand edit, a torn one) is never removed: a rewrite keeps it in its day, after the records, in text order and once, and counts it in `skipped_ledger_lines` on every read. A day holding only such lines stays. A monthly `.jsonl` from an earlier version is still read, split into its days by the first ingest or report whose window touches it, or by a report with no period, then removed; its lines that are not records go to the first day of its month | +| `ledger/.jsonl` | `ingest`, `report` | `report`, `forget` | `StoredUsage` by UTC day of `at`, in time then key order, one call per line. Ingest reads its calls' days and the day on each side (every day when one has no `requestId`), and rewrites only a changed day, whole, by rename; no new bytes, no write. `report` rewrites, folded, the partitions it read that share a call or hold a line that is not a record; a report with no period reads every partition, so after it each call is on disk once. A line that is not a record (a newer version's, a hand edit, a torn one) is never removed: a rewrite keeps it in its day, after the records, in text order and once, and counts it in `skipped_ledger_lines` on every read. A day holding only such lines stays | | `ledger/left-out.txt` | `ingest` | `report`, `forget` | what ingests measured and could not store, by hash, see [Left out](#left-out). cli-v5.5.0 never reads it | | `ledger/repository-ids.json` | `ingest`, `report` | every reader of the ledger, `roots.json` and `branches.jsonl` | `{version: 1, complete, aliases, pending}`: the id an earlier version gave a repository, to its id now, both hashes or root commits. Made from each root `roots.json` names, by its `origin` now. Every read re-keys a Claude Code call stored as `message.id:sessionId:timestamp` and re-ids by this table; the first ingest or report also rewrites `roots.json` and every ledger partition so, then marks it `complete`. A run stopped part way is finished by the next with the same table. A root that could not be read, or was not there, stays in `pending`: every later ingest looks at those roots alone and, when one adds an id, rewrites the stores again. Until then its stored calls keep the earlier id while its new calls get the id of now: two ids for one repository. An earlier version reading migrated data reads it as written; what it stores afterwards is re-keyed, and re-ided when its repository is in the table, at every read. A requestless call it stores then may sit in two days: a report with a period may count that copy, until a report with no period settles the ledger. `branches.jsonl` is never rewritten, so it relies on this table for good: a damaged table is made again from the clones still there, and a snapshot of a repository none of whose clones remains keeps its earlier id, so the calls of now no longer find its declaration. An `undated` hash in `left-out.txt` keeps its earlier key | | `ledger/offsets.json` | `ingest`, `report` | same | per transcript: bytes consumed, size, file identity. Shrunk or changed identity: read whole | @@ -336,6 +316,7 @@ None observable by the CLI. | Key changed by hand, then no hook or ingest before next `on` | interval stays open until observed; calls before observation covered, after turning off included | any session start, prompt or ingest observes it | | Machine clock | judged by transcript timestamp. Stamped inside an interval: covered whenever written; future-dated: covered by an open interval; outside: by stamp alone. File mtime more than a day behind its calls (set back, lagging network fs): not read. Copied transcripts: read only if written after first local `on`, stored only for a clone opted in here | - | | Worktree made and removed between two ingests | `never-seen-alive`, said only by the run that read it: a hook's run says nothing | for a `claude --worktree` session, the session start inside it sees it alive; none for a subagent's worktree | +| A month file (`ledger/.jsonl`) left by an unreleased build between #993 and #1031 | ignored: neither read nor reported | `aidd telemetry forget` removes the whole ledger | | No birth time | a clone reusing a deleted clone's path and inode is taken for it; of two at one directory, the later is told apart only from first seen | - | | Linux refuses `statx` (seccomp, kernel before 4.11) | birth = change time; identity moves on any git dir entry change, `on` included; clone stops being measured and asked | fails closed: `off` closes by key token at path | | Moved or renamed clone | new clone. Old interval open while path empty, closed when another directory appears; nothing stored or asked at new path until `on`. Back at same path, same identity: same clone, still measured | - | diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/mutation-proofs.md b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/mutation-proofs.md deleted file mode 100644 index 909c6fe75..000000000 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/mutation-proofs.md +++ /dev/null @@ -1,460 +0,0 @@ -# Mutation proofs - -For each rule, the test that guards it, the one scripted change that breaks the rule, and what -turned red. Each mutation was applied by script on the file named, the named tests were run, -and the file was restored byte for byte. - -## Phase 3, count (`da23cff3`) - -| Rule | Test | Mutation | Red | -| --- | --- | --- | --- | -| Key with request id | "keys a call by message id and request id" | `:requestId` dropped from the key | yes | -| Requestless key | "keys a requestless call by message id, session id and timestamp", "reads an empty request id as none" | timestamp dropped from the key | yes | -| Synthetic dropped | "drops the synthetic model" | synthetic constant changed | yes | -| Sidechain or agent id is a subagent | "calls a sidechain line a subagent", "calls a line carrying an agent id a subagent" | each check forced false | yes | -| Cache writes summed over message iterations | "sums cache writes…", "does not take the top-level cache write…" | top-level usage only | yes | -| One-hour part unknown when a split is missing | "leaves the one-hour part unknown when any iteration lacks its split" | sum even when a split is null | yes | -| Advisor iterations are their own records | "makes each advisor iteration its own record…" | key `#advisor` without index; wrong iteration type | yes | -| Unrecognised shape reported | "reports a usage line of an unexpected shape…" | identity check disabled | yes | -| Unknown counter is null, never 0 | "leaves a missing counter unknown, never zero…" | `?? 0` on counters | yes | -| Fold: largest total, then earliest instant, then lowest session id | "keeps the record with the largest total", "breaks a tie on total by the earliest time", "compares times as instants", "breaks a tie … by the lowest session id" | each comparison reversed | yes | -| Order independence | "gives the same records whatever the order of the input" | final tie-break or output sort removed | yes | -| Fold across files | "folds a key across files…" | key prefixed by file path | yes | -| Recursive walk, set-asides included | "walks sub-agent directories at any depth", "finds a set-aside whatever its suffix" | no recursion; regex drops `.superseded-*` | yes | -| Byte offsets, unterminated last line, shrunk or replaced file | "counts offsets in bytes…", "does not consume an unterminated last line…", "reads a shrunk file whole", "reads a replaced file whole…" | read from 0; consume to end; size or identity check disabled | yes | -| `CLAUDE_CONFIG_DIR` wins | "is projects/ under the configured directory when there is one" | always the config dir | yes | - -## Phase 4, store (`a80242cc`) - -| Rule | Test | Mutation | Red | -| --- | --- | --- | --- | -| Idempotent re-ingest, deterministic replacement | `stored-usage` "same record comes again", "better snapshot" | held-first order reversed; held always wins | yes | -| Newline guard | snapshot store "crash left no newline" | guard removed | yes | -| Stale lock: dead pid, age; live lock respected; released only by owner | `ledger-lock` tests | each check dropped; unconditional `rm` | yes | -| Consent version, unreadable consent, `AIDD_TELEMETRY=0` | consent and ingest tests | version check dropped; unreadable as absent; refusal removed | yes | -| Never seen alive not stored; persisted resolution after deletion | ingest tests | counted as outside-repo; persisted ignored | yes | -| Remote variants give one id; root-commit fallback; lowest root commit; worktree same id | `repository-identity`, locator integration | normalisations dropped; fallback null; highest root; id from root path | yes | -| Snapshot dedupe; creation time kept; lower-cased config keys | `branch-binding`, snapshot use case | always snapshot; carry-forward removed; lower-casing removed | yes | -| Month partition by UTC instant; positions written last; unchanged partition untouched | `stored-usage`, ledger adapter, ingest | month from raw text; positions dropped; always rewrite | yes | -| Owner-only dir and file | `private-storage` | chmod removed; mode 0644 | yes | - -## Phase 5, declare (`9be2a8f5`) - -| Rule | Test | Mutation | Red | -| --- | --- | --- | --- | -| Default branch and detached never bound | `branch-role`, declare use case, shared fixture | default check false; detached as working | yes | -| Latest declaration wins from its own time, tie to the later line | `binding-resolution`, fixture `two-declarations-at-one-instant-the-later-line-wins` | `time <= at` dropped; `>=` to `>` | yes | -| None declaration | resolution and `task-declaration` tests | `none` ignored; writer sets `none: false` | yes | -| Snapshot at once; keys survive rename; stale ticket removed | declare use case tests | snapshot call removed; key prefix changed; unset skipped | yes | -| Fixture field drift | fixture "is written with the fields the CLI writes" | writer emits `declaredAt` | yes | - -## Phase 6, ask (`165c7c0b`) - -| Rule | Test | Mutation | Red | -| --- | --- | --- | --- | -| Claude-only guard | claude-only fixture cases, Codex and Copilot payloads, Codex rollout under `projects/` | session-id equality, `projects` check or rollout rejection removed | yes | -| Presence | presence matrix | `ATTENDED=1` or `sdk*` rule dropped; absent entrypoint accepted | yes, for the rule of the time. The `sdk*` rule is replaced by the surfaces seen: see "Presence, observed on each surface" | -| ~~Consent~~ | ~~consent fixture, `AIDD_TELEMETRY`~~ | ~~version, env refusal or worktree fallback dropped~~ | superseded: this row described the deleted file-based consent (`.aidd/config.json`, a version). Consent is the clone's git config now: see Fix round 1 phase 6 and phase 8 | -| Default and detached pass | branch-role fixture | forced `working` | yes | -| Answer-path check | aidd absent, no `telemetry task` | `canAnswer` always true | yes | -| Strict intercept | 22-prompt injection test | forbidden-character, unknown-option or word-only check removed | yes | -| Clear and fork carry | clear, chain, other pid, unbound, pid before boot | pid ignored; bound check skipped; boot guard dropped | yes | -| No carry on resume, startup, compact | source matrix | those sources carried | yes | -| Silent catch-up | catch-up test | stdout printed; `--quiet` dropped | yes | -| Declaration suppressed and marked | typed-declaration test | `suppressOriginalPrompt` false; `--by` removed | yes | - -## Phase 7, attribute and report (`cbc2ceb9`, `a53833d4`) - -| Rule | Test | Mutation | Red | -| --- | --- | --- | --- | -| Session declaration from its time; mid-session redeclaration | `attribution.unit.test.ts` "applies a later declaration from its own time only" | time ignored; carried floor uses latest declaration | yes | -| Carried session moved whole by its first declaration | "is moved whole by its first declaration, the work before it included" | floor ignores declarations | yes | -| Carry covers the whole carried session | fixture `carried-before-the-carry-falls-to-branch` | carry floor −∞ | yes | -| Branch binds after creation; declared after the work; repository and name | attribution branch tests | creation check removed; from declaration time; name-only lookup | yes | -| Reused branch name keeps generations | "keeps the work of an earlier branch of that name on its own task" | latest snapshot only | 7 red | -| `declared-none`; session beats branch; subagent inherits | attribution tests | none as no-binding; order swapped; session lookup skipped | yes | -| Unknown counter never summed as zero | `usage-report.unit.test.ts` | null no longer counted | yes | -| Every axis reconciles | `usage-report.property.unit.test.ts` | ticket rows without ticket dropped | yes | -| Identity shape; oldest transcript date | identity and ingest tests | one-field rule relaxed; result null | yes | - -## Phase 8, opt in, opt out, forget (`8f9a757f`) - -| Rule | Test | Mutation | Red | -| --- | --- | --- | --- | -| Hook line and delegate go together | lifecycle "keeps the script while a lefthook job still calls it…", husky variant | `callers.length === 0` to `>= 0` | 2 red | -| Foreign hook content byte for byte | "keeps a hook's other content byte for byte, and its mode" | lines trimmed on rejoin | yes | -| Forget preview changes nothing | "previews and changes nothing on disk, not a byte nor an mtime" | `!confirmed` to `confirmed === undefined` | yes | -| ~~Offsets reset on `on`~~ | ~~"is stored once opting in, though ingest had read past it"~~ | ~~`resetPositions()` removed~~ | superseded by fix round 4: `on` reads nothing back and resets no offsets; the test now asserts the opposite | -| ~~`on` writes version 2; `off` keeps version~~ | ~~switch tests~~ | ~~`version` dropped; spread dropped~~ | superseded: the file-based switch is deleted. `on` writes `aidd.telemetry=2` and `off` writes `off` in git config: see Fix round 1 phase 8 | -| ~~Remembered roots re-resolved~~ | ~~"is stored for a directory that is gone, whose refusal was remembered"~~ | ~~`resolutions.save` removed~~ | superseded by fix round 2: `on` rewrites no remembered refusal, a deleted directory is judged by its clone | - -## Phase 9, skills and journey (`2486849d`, `e6b04615`) - -| Rule | Test | Mutation | Red | -| --- | --- | --- | --- | -| Skills name only real commands, flags, axes | `telemetry-skills-name-real-commands.test.js` | `report` to `reprt`; `--yes` to `--yess`; `--axis tasks`; `--axis tickets` in `frame.md` | yes, each | -| Skills never name hooks or V1 | same test | a "hook" word added | yes | -| Journey: carry, presence, gate, carries in report | `telemetry-journey.e2e.test.ts` | carry never written; presence false; gate skips presence; carries ignored | yes, each | - -Phase 8 rows were later revised by the consent move (fix round 1): consent is git config now, and its mutations are below. - -## Fix round 1, phase 4 (store, ingest, consent) - -| Rule | Test | Mutation | Result | -| --- | --- | --- | --- | -| Partitioning is linear in the records | `stored-usage.unit.test.ts` "copies each record a bounded number of times" (counts array iteration, deterministic) | `partitionByMonth` copies the month array per record again (`[...(held ?? []), record]`) | 1 red, that test (2,003,001 elements touched at n=2000 before the fix) | -| An upsert names the months it changed, the old month of a replaced record included | `stored-usage.unit.test.ts` "names the months it changed" | loop over `[record]` instead of `[record, was]` | 1 red, that test | -| Ingest tells the ledger which months to write | `ingest-usage-use-case.unit.test.ts` "rewrite only the month a new call falls in" | `save(merged.records)` without the months | 1 red, that test | -| A save told the months reads and writes only those | `usage-ledger-adapter.integration.test.ts` "rewrites only the months it is told changed" | the filter by told months removed | 1 red, that test | -| A snapshot reads and appends under the bindings lock | `snapshot-bindings-use-case.unit.test.ts`, `bindings-lock-adapter.integration.test.ts` "leaves one snapshot when ... at once", `declare-task-use-case.unit.test.ts` | `append` called without `lock.exclusively` | 3 red: the snapshot lock order, the concurrent-snapshot test (3 writers, more than 1 snapshot), the declare order | -| A declaration's session line is appended under the bindings lock | `declare-task-use-case.unit.test.ts` (event order, default-branch order) | `sessions.append` outside the lock | 2 red, those tests | -| A declaration never waits on the ledger lock | `telemetry-task.e2e.test.ts` "declares at once while an ingest holds the ledger lock" | not mutated: before the fix the test timed out at 5 s (declaration blocked, 60 s lock wait); the declare use case no longer receives the ledger | red on the pre-fix binary, green after | -| ~~`on` lifts a remembered refusal by repository, a deleted linked worktree included~~ | ~~`resolution-consent.unit.test.ts`, `telemetry-switch-use-cases.unit.test.ts`~~ | ~~`ofRepository = false`~~ | superseded by fix round 2: the lifting by `repository_id` leaked consent across clones and is deleted | -| ~~`on` passes the repository id~~ | ~~same use-case test~~ | ~~`repositoryIdOf(located)` replaced by `null`~~ | superseded by fix round 2, with the lifting | -| A declaration is read at the call's time within a branch generation, the first one covering from creation | `attribution.unit.test.ts` three new "redeclaration" tests | `generationAt` returns `generation.at(-1)` (latest snapshot) | 5 red (recorded 3 before; re-run in fix round 2 over `tests/contexts/telemetry`: the three new redeclaration tests, "speaks with the latest snapshot of one declaration…" and "lets a snapshot with no declaration time cover from the creation…") | - -## Fix round 1, phase 6 (hooks) - -| Rule | Test | Mutation | Result | -| --- | --- | --- | --- | -| A prompt that only starts like a declaration, or holds shell syntax, goes to the ordinary gate and spawns nothing | `aidd-telemetry-hooks.test.js` "injection" and "only starts like a declaration" | an intercepted prompt that does not parse blocks "not understood" again | 2 red (since narrowed by fix round 2: only a prompt that attempts a declaration is told so) | -| The `cmd.exe /s` line has an outer pair of quotes | same file, "cmd.exe /s command line" | `cmdLine` returns `"shim" args` | 1 red | -| The hook's home is the OS home, as the CLI's | same file, "the telemetry directory's home" | `home ?? set(env.HOME) ?? os.homedir()` | 1 red | -| A committed `.aidd/config.json` is not consent | same file, "a committed .aidd/config.json is not consent" | the hook also accepts a committed `telemetry.version === 2` | 1 red | -| `AIDD_TELEMETRY=0` refuses before git is asked | same file, "AIDD_TELEMETRY=0 refuses" and the session-start test | the environment check removed | 2 red | - -## Fix round 1, phase 8 (consent, leftovers, forget) - -| Rule | Test | Mutation | Result | -| --- | --- | --- | --- | -| Nothing in `.aidd/config.json` is consent (CLI) | `git-consent-adapter.integration.test.ts`, `telemetry-lifecycle.integration.test.ts` | the adapter falls back to a committed `telemetry.version === 2` | 2 red | -| `AIDD_TELEMETRY=0` refuses (CLI) | `declare-task-use-case.unit.test.ts`, `ingest-usage-use-case.unit.test.ts` | each refusal check removed in turn | 1 red each | -| `on` removes the previous version's block, and deletes the file only when emptied | `telemetry-switch-use-cases.unit.test.ts`, `telemetry-lifecycle.integration.test.ts` | the legacy cleaning replaced by "none" | 6 red | -| `off` writes `off` | `telemetry-switch-use-cases.unit.test.ts` | the write removed | 1 red | -| `forget` unsets the consent key | `repository-declarations-adapter.integration.test.ts` | the consent key left out of the unset loop | 1 red | -| `forget` finds a repository that only opted in | `forget-telemetry-use-case.unit.test.ts` | consented resolutions no longer added | 1 red | -| `on` ends with the next step | `telemetry-lifecycle-display.unit.test.ts` | "Next:" line reworded | 1 red (after the test was tightened: the first version of the test survived this mutation) | -| The unknown-counter message says what happens to the call | the two display tests | message back to "not recognised and not counted" | 2 red | - -Rows before fix round 1 record "yes" and no count: they were not counted then, and are not re-run here. Only the row corrected above was re-run. - -Not mutated: removing `outside-repo` and `root-unresolved` from the unattributed reasons (a -type-level change: the exhaustive `Record` makes a reinstated reason fail the -typecheck), and the documentation corrections. - -## Fix round 2 - -Every row below was re-run at the final HEAD of the round: the mutation applied by script to the -file named, the scope run (`npx vitest run tests/contexts/telemetry tests/presentation` plus -`tests/contexts/tools` and `tests/contexts/framework/application/plugin` for the Codex row, and -`node --test scripts/__tests__/aidd-telemetry-hooks.test.js` for the hook row), and the file -restored (checked equal to the original by the script). Counts are failing tests. - -| Rule | Test | Mutation | Result | -| --- | --- | --- | --- | -| `on` lifts nothing: a clone's consent never reaches another clone sharing the remote or the root commit | `telemetry-switch-use-cases.unit.test.ts`: (b) "keeps refusing a deleted clone of the same remote that never opted in", (c) "keeps refusing a clone of the same remote that ran off, deleted or not" and "keeps refusing what a clone made while off, though it ran on first and was then deleted", (e) "keeps refusing a deleted copy with no remote that shares the root commit", and "resets the offsets … rewrites no refusal" | `rememberOwnRoot` also marks every remembered resolution of its `repository_id` consenting (the lifting reintroduced) | 5 red, those five. Before the fix (b), (c) and (e) were red for the same reason | -| A deleted directory is judged by its clone's git config read live; the remembered flag only once the clone is gone | `ingest-usage-use-case.unit.test.ts` "a directory that is gone is judged by the clone it belonged to"; `telemetry-switch-use-cases.unit.test.ts` "stores a deleted linked worktree of the clone that opted in"; `telemetry-lifecycle.integration.test.ts` "is stored for a directory that is gone, whose refusal was remembered" | `fromMemory` always takes the clone as gone (remembered flag only) | 6 red | -| `off` withdraws what was remembered of its own clone, so a deleted clone is not judged by a stale yes (my reading of "the remembered consent is used only when the clone is gone": the memory has to be right for that case) | `telemetry-switch-use-cases.unit.test.ts` "keeps refusing what a clone made while off, though it ran on first and was then deleted", "withdraws what was remembered of the clone's other directories, and of no other clone", "writes under the ledger's lock …" | the `forgetConsentOfClone` call removed from `off` | 3 red | -| `off` lowers the clone's own entries only | same file, "withdraws what was remembered of the clone's other directories, and of no other clone" | the clone check removed in `forgetConsentOfClone` | 1 red | -| `on` remembers its own root, so `forget` finds the clone though no session ever ran there | `telemetry-switch-use-cases.unit.test.ts` "remembers its own root, seen alive, so forget can find the clone later"; `telemetry-lifecycle.integration.test.ts` "unsets the consent of a clone where `on` ran and no session ever did, and of one that ran off" (the in-process twin of the e2e "leaves no consent behind in a clone where `on` ran and no session ever did") | `rememberOwnRoot` records nothing | 5 red, the integration twin among them. The e2e on the built binary was not run against this mutation | -| `forget --yes` clears the consent of every clone recorded, those `on` recorded included | `forget-telemetry-use-case.unit.test.ts` "clears the consent of a clone `on` remembered, though no session ever ran there" and the rest of the clone-keyed suite | forget skips the clones recorded as consenting | 12 red (the whole clone-keyed suite builds its clones that way) | -| `forget` locates a directory remembered before clones were recorded whatever consent it found | `forget-telemetry-use-case.unit.test.ts` "finds the clone of a directory remembered as refusing before clones were recorded, whose key says off" | a legacy entry is skipped unless it consented | 1 red | -| A mistyped declaration is blocked with the grammar and the quoting hint, and runs nothing | `aidd-telemetry-hooks.test.js` "a declaration mistyped is blocked with the grammar and the quoting hint, and runs nothing" | `attemptsDeclaration` never consulted (always fall through) | 1 red, that test; the injection test and "only starts like a declaration" stay green | -| Ingest rewrites a month that holds a line that is not a record | `ingest-usage-use-case.unit.test.ts` "also rewrites a month that holds a line that is not a record…", "repairs a damaged month even when every call it read was already held" | `damagedMonths` left out of the months saved | 2 red | -| The branch config is read inside the bindings lock | `snapshot-bindings-use-case.unit.test.ts` "reads the branch config, the latest snapshots and appends … inside the bindings lock" | the config read once, before the lock, and passed in | 1 red | -| Each lock names itself in its timeout message | `bindings-lock-adapter.integration.test.ts` "names itself, and its own file, when it times out" | the bindings adapter stops passing its name | 1 red | -| The `total` axis shows its row once | `telemetry-report-display.unit.test.ts` "shows the whole once on the total axis, and a Total row under every other axis" | the `Total` row always added | 1 red | -| The Codex trust notice no longer speaks of a run journal | `codex.unit.test.ts`, `plugin-add-mcp.unit.test.ts` | the old sentence restored | 3 red | -| An unreadable consent is worded as the clone's git config | `telemetry-display.unit.test.ts` "words an unreadable consent as the clone's git config…" | the old `.aidd/config.json` wording restored | 1 red | -| `on` and `off` speak of the clone | `telemetry-lifecycle-display.unit.test.ts` "says measurement is on for this clone, …" | "this repository's linked worktrees" restored | 1 red | -| (re-run) a reused branch name keeps the declaration in force at the call | `attribution.unit.test.ts` "a branch name used again" | `generationAt` returns `generation.at(-1)` | 5 red (the figure the review observed; the 3 recorded before was wrong) | - -Not mutated: the help wording (`aidd telemetry on`, `off`, `ingest`) is pinned by the help golden -(`help-surface.e2e.test.ts`); `target.md`, `codebase-map.md`, `telemetry.md`, `usage-contract.md` -and the comment rewording are documentation. - -Not seen red before the fix, proved by mutation afterwards: the forget tests, the snapshot-in-lock -test, the unreadable-consent wording test and the Codex notice (the test and the wording were -changed together). Seen red first: the `on`/`off` wording test, the resolver and `on` tests ((b), -(c), (e), the worktree), the `off` tests, the legacy forget test, the hook test, the torn-line -tests, the lock-name test and the duplicate-row test. - -## Fix round 3 - -One predicate, one table. `cli/tests/contexts/telemetry/consent-scenarios.integration.test.ts` runs real git repositories in temporary directories, the real adapters and a real ingest, and states the calls the ledger ends up holding. It was written first, against the code of `309f7393` and nothing newer (only `wireTelemetry` and each use case's `execute`). - -Before the fix, per row (`npx vitest run … --reporter=verbose` at `309f7393`): - -| Row | Scenario | Before the fix | After | -| --- | --- | --- | --- | -| 1 | (b) deleted clone that never opted in, then `on` in another clone of the remote | pass (fixed in round 2) | pass | -| 2 | (c) clone that ran `off`, then `on` in another clone | pass (fixed in round 2) | pass | -| 3 | (e) deleted copy, no remote, root commit shared, carrying the source's consent | **fail**: `c1` stored while the copy was alive | pass | -| 4 | deleted linked worktree seen alive before `on`, then `on` in its clone | pass | pass | -| 5 | same repository cloned again at the path, then `on`: the old clone's deleted worktree | **fail**: `pw` stored | pass | -| 6 | unrelated repository cloned at the path, then `on`: the old clone's deleted worktree | **fail**: `pw` stored | pass | -| 7 | the old clone's own calls, cloned again at the path, `on` there | **fail**: `po` stored | pass | -| 8 | `on`, calls, `off`, calls, `on`, calls, ingest after each batch | **fail**: the middle call stored | pass | -| 8b | the same, one ingest at the end (added: the result must not depend on ingest timing) | **fail**: the middle call stored | pass | -| 9 | `on`, a manual `off` an ingest observes, calls, clone deleted | **fail**: the call after the observation stored | pass | -| 10 | first `on`, history from before it | pass | pass | -| 11 | `AIDD_TELEMETRY=0` | pass | pass | -| 12 | `forget --yes` after `on`, no session | **fail**, on its precondition: `on` wrote no `ledger/consents.jsonl`. The `forget` and the two assertions after it were not reached at the old code; the file assertion could only have passed vacuously there, the file never existing | pass | - -Rows 1, 2, 4, 10 and 11 guard behaviour that was already right: they stay green, and the mutations below show they can go red. Every "not stored" row holds a positive control, a call that must be stored, so none passes because ingest stored nothing. A clone "deleted" in a row is moved aside: a deleted directory's inode may be given to the next one on some file systems, and a row about a clone told apart from its successor must not depend on that. - -Only the table was written before the code. The unit and integration tests under it were written with the code and are proved by the mutations below, run afterwards. - -Every row below: the mutation applied by script with an exact-anchor check, `npx vitest run tests/contexts/telemetry tests/presentation` (the Node suite for the hook row), the file restored and checked byte-equal. Counts are failing tests. - -| Rule | Mutation | Result | -| --- | --- | --- | -| A clone is told from another at its path by its identity | identity ignored, read by path only (`sameClone` and `cloneKey` compare the path alone) | 22 red, including rows 5, 6, 7 | -| A call is judged at its own time | the intervals ignored: any `on` of the clone covers every call | 13 red, including rows 2, 8, 8b, 9 | -| A call made while off is never stored | the off window included: an `off` closes nothing | 19 red, including rows 2, 8, 8b, 9 | -| The end of an interval is excluded | `at <= to` | 4 red | -| The start of an interval is included | `from < at` | 3 red | -| Fail closed when the platform gives no inode | the `ino === 0n` check removed in `identityFromStat` | 1 red: `clone-identity.unit.test.ts` "is none where the platform reports no inode". No table row reaches it: a real macOS stat never returns an inode of 0, so the check is pinned at the seam, and the locator and the consent adapter take an identity reader that the tests make blind | -| Ingest looks at every clone whose consent is open | `observeOpenConsents` not called | 3 red, including row 9 | -| A clone found gone ends its consent | the close on `gone` removed | 2 red | -| A clone whose key is not `2` ends its consent | the close on `absent` removed | 4 red, including row 9 | -| A live clone whose key is not `2` stores nothing, whatever its intervals | `state === "absent"` no longer refuses: live key not required | 8 red | -| The latest clone born by the call answers for a directory | always the latest | 6 red, including row 7 | -| The first clone answers for a call older than every clone | always the first | 4 red, including row 7 | -| `off` ends the interval | the close removed | 6 red, including rows 2, 8, 8b | -| `on` opens the interval | the open removed | 23 red, all thirteen rows | -| `forget` names a gone clone only if it consented | any gone clone named | 2 red | -| Ingest saves the damaged month (round 2's weak test, now asserting which months were saved) | `damagedMonths` dropped from the saved set | 2 red: "repairs a damaged month even when every call it read was already held" and "also rewrites a month that holds a line that is not a record". Observed with the new assertion on the saved months taken out of the test and the same mutation: 1 red, the second test only, which is the review's own line "M5: `damagedMonths` dropped from the saved set only gives 1 red (recorded 2; see 🟢 test gap)" | -| A mistyped declaration says how to send it as an ordinary prompt | the message line removed | 1 red: "a declaration mistyped is blocked…". A second test, "a question mark makes it an ordinary prompt", checks that the way out the message gives works; it is not mutation-proved, because dropping `?` from the `attemptsDeclaration` regex leaves it green (the tokeniser already refuses a `?`) | - -Survivors left in the files of this round, none believed to be a gap: `clone-identity-reader.ts` `{ bigint: true }` (an inode above 2^53 cannot be made on the machine this ran on), `clone-identity.ts` `<= 0n` against `< 0n` (equivalent: 0 maps to 0), `birthtimeMs` type check against the finite check (equivalent), `directory-resolver.ts` filter of the current owner (the owner list is the same set), `""` fallbacks on a missing file. - -Mutation score of the `telemetry` scope at the head of the round: 96.4 (3,343 detected, 126 undetected), floor 96. - -Not mutated: `usage-contract.md`, `target.md`, `codebase-map.md`, `telemetry.md` and the comments; the per-OS notes on the identity in `clone-identity.ts` (macOS observed; Linux and Windows read from Node's `fs.Stats` documentation and libuv's `src/unix/fs.c`, `src/unix/linux.c` and `src/win/fs.c`, v1.x, and not run). - -## Fix round 4 - -The user decided: `on` no longer catches up history, and a call is stored only if its clone's consent was open at the call's time. The rule is in `usage-contract.md` § Consent, written first. Consent is now a key `2:` together with an interval in `ledger/consents.jsonl` (`{token, clone, open}` and `{token, close}`), closed by `off`, by ingest and by the hooks. - -### The table, before and after - -`consent-scenarios.integration.test.ts` was extended before any code, using only `wireTelemetry`, each use case's `execute` and the real `prompt-gate.cjs`, so it ran at `d67ed04e` (`npx vitest run tests/contexts/telemetry/consent-scenarios.integration.test.ts --reporter=verbose`). Rows 1, 2, 4 and 9 had positive controls dated before their `on`; under the new rule those controls are no longer stored, so each control was moved after its `on` (row 4 was reshaped: the worktree is seen alive after `on`, then removed, and its next call is judged by the remembered clone). Row 10 flips, and 10b (the same, never ingested before `on`) was added. The hook rows run the real `prompt-gate.cjs` with a stand-in `aidd` that answers `telemetry task --help`, so a row that expects no block holds a control that does block. - -Before the fix, 13 rows failed and 13 passed. After it, all 26 pass. - -| Row | Scenario | Before | After | -| --- | --- | --- | --- | -| 1 | (b) deleted clone never opted in, then `on` in another clone of the remote (control after `on`) | pass | pass | -| 2 | (c) clone that ran `off`, then `on` in another clone (control after `on`) | pass | pass | -| 3 | (e) deleted copy, no remote, root commit shared, carrying the source's consent | pass | pass | -| 4 | worktree of an opted-in clone seen alive, removed before the next ingest | pass | pass | -| 5 | same repository cloned again at the path, then `on`: the old clone's deleted worktree | pass | pass | -| 6 | unrelated repository cloned at the path, then `on`: the old clone's deleted worktree | pass | pass | -| 7 | the old clone's own calls, cloned again at the path, `on` there | pass | pass | -| 8 | `on`, calls, `off`, calls, `on`, calls, ingest after each batch | pass | pass | -| 8b | the same, one ingest at the end | pass | pass | -| 9 | `on`, a manual `off` an ingest observes, calls, clone deleted (control after `on`) | pass | pass | -| 10 | history before the first `on` is **not** stored (changed) | **fail**: `a0` stored | pass | -| 10b | the same, never ingested before `on` | **fail**: `a0` stored | pass | -| 11 | `AIDD_TELEMETRY=0` refuses everything | pass | pass | -| 12 | `forget --yes` after `on`, no session | pass | pass | -| h1 | old clone never ingested while it lived, re-clone of the same repository at its path, `on` | **fail**: `p1` stored | pass | -| h1b | the same, an unrelated repository at the path | **fail**: `p1` stored | pass | -| h1d | outside any repository refused, `git init`, `on`: nothing comes back | **fail**: `d1` stored | pass | -| h15 `off` | `on`, c1, manual `git config aidd.telemetry off`, a headless hook prompt, c2, `on`, c3, one ingest: c1 and c3 stored, c2 not | **fail**: c2 stored | pass | -| h15 `--unset` | the same with `--unset` | **fail**: c2 stored | pass | -| h15b | the same with no hook between: c2 stored, a documented residual limit | pass | pass | -| h2 | `off`, the clone moved, `on`: calls made while off not stored | **fail** | pass | -| reverse | `on`, c1, `off`, one ingest at the end: c1 stored | **fail**: nothing stored | pass | -| reverse 2 | on/off/on/off, one ingest at the end: c1 and c3 stored | **fail** | pass | -| forget-on | `forget` then `on`: earlier calls not stored | **fail** | pass | -| hand key | key set by hand (`2` and `2:forged`) and a `cp -R` copy: nothing stored, the real gate does not block; the opted-in original stores and blocks | **fail**: the gate blocked the hand-set clones | pass | -| damaged | a damaged line in `consents.jsonl`: nothing more stored, `unreadable-consent` counted | **fail**: `c2` stored | pass | - -Every "not stored" row holds a positive control (a call that must be stored), so none passes because ingest stored nothing. The "before" column of rows 10 and later is as observed; it was not re-derived from a diagnosis. Rows 1 to 9, 11 and 12 pass on both sides because they guard behaviour that was already right (their mutations below show they can go red). - -### Mutations - -Each applied by script with an exact-anchor count of 1 (`scratchpad/mut.py`), running `npx vitest run tests/contexts/telemetry tests/presentation` and `node --test scripts/__tests__/aidd-telemetry-hooks.test.js`, then the file restored and checked byte-equal. Counts are failing tests, vitest then node. - -| Rule | Mutation | Result | -| --- | --- | --- | -| The hook needs an interval, not the key alone | `decideConsent` grants on the key's token alone (intervals ignored; any key counts) | 1 + 2: row "hand key"; the fixture's `hookConsent` cases, "a cp -R copy carries the key…" | -| A call is judged at its own time | the CLI `covers` ignores the time of an interval | 34 + 0 | -| A call made while off is never stored | `off` closes nothing (off window included) | 9 + 0, including rows 2, 8, 8b, h2, reverse, reverse 2 | -| The hook ends an interval its key no longer names | the hook's close not written | 2 + 2: rows h15 `off` and `--unset`; "a clone whose key was turned off by hand has its interval closed…", "the session start and the catch-up close it too" | -| The hook knows its clone by real path | the real path check removed in `decideConsent` | 1 + 2: row "hand key" (the `cp -R` copy); the fixture; "a cp -R copy carries the key…" | -| A damaged log refuses storage | the CLI's damaged check removed | 2 + 0: row "damaged"; "stores nothing for any clone while a line of the consent log is damaged" | -| A damaged log grants the hook nothing | the damaged check removed in `decideConsent` | 0 + 1: the fixture's `hookConsent` cases only. The behaviour test stays green because `consentGranted` refuses a damaged log before `decideConsent` is reached: the two checks overlap, and only the pure function is pinned by the fixture | -| Nothing before `on` is stored | an `on` interval opened at `-Infinity` (`new Date(-8.64e15)`) | 14 + 0, including rows 10b, h1, h1b, h15, reverse 2, h2, forget-on | -| A token closed twice ends at the earliest close | the latest close | 2 + 0 | -| The key goes before the interval | the interval appended before the key | 1 + 0: "writes the key before the interval…" | -| Ingest ends an interval its key no longer names | the live key is not compared with the interval's token | 15 + 0, including row 9 | -| An interval that is still named stays open | every observed interval closed | 8 + 0 | -| The prompt gate observes the key before it asks if a person is present | presence asked first | 2 + 1: rows h15 `off` and `--unset`; "a clone whose key was turned off by hand has its interval closed, now, even with nobody present" | -| The end of an interval is excluded | `at <= to` | 14 + 0 | -| The start of an interval is included | `from < at` | 7 + 0 | -| A clone is told from another at its path by its identity | identity ignored, `cloneKey` is the path | 20 + 0 | -| An unreadable git config refuses | the check removed | 2 + 0 | -| A clone found gone ends its interval | a gone clone counted as still naming the token | 2 + 0 | -| `on` ends the stale intervals of its clone | the close removed | 1 + 0 | -| `task` applies the same test as the hooks | the interval check removed in `ConsentedRepositories` | 1 + 0 | -| `off` asks the log only for a clone it can identify | `if (located.clone !== null)` always true | 1 + 0 | -| A clone that never asked, with a damaged log, is "no consent", not "unreadable" | the early `absent` refusal removed in `ConsentedRepositories` | 1 + 0 | - -Mutation score of the `telemetry` scope at the head of the round: 96.2 (floor 96). Stryker survivors in the files of this round, none believed to be a gap: `consent-history.ts` `sameKeys` with `some` for `every` (the parse that follows rejects the line either way), `directory-resolver.ts` filter of the current owner (the owner list is the same set), `telemetry-on-use-case.ts` `held !== null` and the `value` check of the re-read key (a null token names no interval; the unreadable case was refused before), `consented-repositories.ts` `status: "open"` text and the `value` check (nothing reads them but the declare use cases, which only test for a refusal). - -### Residual limits recorded in the contract - -A manual change of the key between two observations (row h15b), the clock (a call is judged by its transcript timestamp), a worktree made and removed between two ingests, no birth time, a moved clone, a reused process id, and `AIDD_TELEMETRY=0` stopping the hooks too. See `usage-contract.md` § Consent, Residual limits. - -### Deviations and decisions of the round - -- `ownerAt` is kept, with its "else the first" fallback: with intervals it grants nothing on its own (the owner's interval must cover the call, and none opens before an `on`), but a clone with no birth time is known only from the day ingest first saw a directory in it, and a call from a subdirectory made before that day is still its call. -- `rememberOwnRoot` is kept (renamed `remember-own-root.ts`): it records a root seen alive so `forget`'s `unlocated` count is right for a clone where `on` ran and no session did. It is not a consent flag. -- `repository-consent.ts` (a one-line wrapper) is deleted; `resetPositions` is removed from the ledger port and adapter. -- `aidd telemetry task` (and `show`) now require the same test as the hooks (key names an interval open for the clone); the brief named the hooks only. -- The hook does nothing under `AIDD_TELEMETRY=0`, closes included: a residual limit. -- A hook closes all open intervals of its clone's real path whose token the key does not name, including a stale one next to a named one. - -## Fix round 5 - -Each mutation was applied by script (exact anchor, count 1) and run against `npx vitest run tests/contexts/telemetry tests/presentation` (CLI, 1623 tests) and `node --test scripts/__tests__/aidd-telemetry-hooks.test.js` (hooks, 61 tests). The file was restored from a copy after each. Counts are `CLI failed + hook failed`. - -| Fix | Mutation | Red | -| --- | --- | --- | -| 1 `on` refuses a damaged log | the guard `if (log.damaged) return "damaged"` made dead (`&& false`) | 2 + 0: rows "damaged on" and "damaged recovery" | -| 1 `off` refuses a damaged log | the same guard in the `off` use case | 1 + 0: row "damaged off" | -| 2 the hook compares the full identity | `sameIdentity(...) \|\| true` in `decideConsent` | 1 + 2: row "copy at the path", and the fixture cases for another inode, device and birth time | -| 3 an unstattable clone never aborts | `readCloneIdentity` rethrows instead of `unidentified` | 3 + 0: the reader test, the locator test, row "unstattable" | -| 4 a clone out of reach stays open | a clone with nothing at its path is closed again | 2 + 0: row "unmounted" and the ingest unit test | -| 5 a broken `.git/config` is unreadable | the locator answers `outside-repository` again | 2 + 0: the locator test and row "broken config" | -| 8 the journey no longer backdates consent | `on` opens its interval one hour late, built and run on `tests/e2e/telemetry-journey.e2e.test.ts` | 1 failed of 3: "total: expected 1 to be 5" | - -Fixes 6 (a comment) and 7 (a removed export) change no behaviour, so no test can go red for them. For 7, `git grep -n consentOf -- plugins scripts` finds nothing, and the hook test now uses `tokenOfKey`. - -Mutation score of the `telemetry` scope after the round: 96.2 (floor 96). - -### Deviations and decisions of the round - -- Recovery from a damaged log is `forget --yes` then `on`, by `on` and `off` refusing and by the advice; a hand edit that removes the damaged line is outside the contract and, as the review's script A2 does it (and then re-reads the transcripts), still stores the off window. The tool no longer suggests it. A latch that outlives a hand edit would be a design change and was not made. -- A new `consent-closed` reason joins `coverage.not_stored`, and `coverage.consent_log_damaged` is added to the version 1 envelope. -- The locator answers a new `unreadable` status for a directory it cannot look at and for a `.git` git cannot read; `outside-repo` is left for what is not a repository. - -## Fix round 6 - -Each mutation was applied by script (exact anchor, count 1), run against the named test file, and the file restored from its bytes and compared equal. Counts are the failures observed. - -| Fix | Mutation | Red | -| --- | --- | --- | -| 1 smoke order | not mutated: the run before the fix is the proof. `telemetry off` after `forget --yes` printed "Measurement was not on for this clone." and the step failed ("PASS: 128 FAIL: 1" in the review). After the reorder, "PASS: 130 FAIL: 0" | n/a | -| 2 the CLI parser ignores an unterminated last line | `terminated.pop();` removed in `parseConsentRecords` | 2 in the fixture test (`torn-last-line-is-not-yet-written`, `whole-event-without-its-newline-is-not-yet-written`), 1 in the row "torn tail" | -| 2 the hook parser ignores it | `terminated.pop();` removed in `parseConsentLog` | 1 in the hook test "the consent log follows the fixture" (it loops the cases) | -| 2 the CLI writer cuts the torn tail before appending | `await this.dropUnterminatedTail();` removed | 1 in the adapter test, 1 in the row "torn tail" | -| 2 the hook writer cuts it | `{ dropTornTail: true }` turned `false` | 1 in the hook test "a torn last line leaves the clone measured…" | -| 3 only a hook declaration waits less | `lockWaitFor` never answers a wait | 1 in "waits for the bindings lock less than the hook does…" | -| 3 the snapshot waits less too | `lockWaitFor(declaration.by)` dropped in `BranchDeclarations.declare` | 1, same test (waits `[8000, undefined]`) | -| 3 the session line waits less | the wait dropped in `DeclareTaskUseCase.execute` | 1, same test | -| 3 the lock honours a wait | the `waitMs` override dropped in `DirectoryLock.exclusively` | 1 in "gives up after the wait a caller asks for…" | -| 3 the adapter passes it on | `wait?.waitMs` replaced by nothing in `BindingsLockAdapter` | 1, same test | -| 5 `no-consent` wording | the display text put back to "has not opted in" | 1 in "lists the calls it did not store, by reason" | - -Fixes 4 and 6 are documents; no test can go red for them. - -### Deviations and decisions of the round - -- The writers' newline guard (`appendRecord`, `PrivateStorage.append`) was not enough for fix 2: it ends a torn tail with a newline, which turns it into a terminated line that is not an event, so the next `on`, `off` or hook close would have damaged the log. The test failed that way before the change (hook: `Unterminated string in JSON` reading the log back). The two consent writers now cut the unterminated tail before appending (`ftruncate` in the hook, a `replace` in the CLI). Every other file still gets the guard. The cut is not atomic with a concurrent append: a line another process appends between the read and the cut can be lost, a window of a few milliseconds that only opens on a log already torn. -- A last line that would parse but has no newline is ignored too: a write is one line and its newline, so one without the other is not finished. -- The hook-intercept wait is 8 s, not 10 s, because a declaration takes the bindings lock twice (the session line, then the branch snapshot): at most 16 s of waiting under the hook's 20 s. - -## After the first CI runs - -### A birth equal to the change time is a birth - -The first rule took a birth equal to the change time as no birth, in milliseconds, then in nanoseconds. Both were wrong, and the row "copy at the path" kept failing on Linux now and then, with `expected [ 'a1', 'b1' ] to deeply equal [ 'b1' ]`. Measured in Docker (`node:22-bookworm`, aarch64, kernel 7.0, OrbStack, overlay on btrfs): - -- **Cause, observed.** Two failing runs, instrumented after the ingest: the copy at `a/.git` was stored in `roots.json` with `birthtimeMs: 0`, and its stat had `birthtimeNs` equal to `ctimeNs` (`1791709014192859969` both). With no birth, the copy answers only from ingest's first sighting, so the call made before it went to the original clone, whose consent was open. -- **Rate.** `cpSync` of a fresh git dir, 2000 times: birth `0` never, birth equal to the change time to the nanosecond 1529 times, in the same millisecond but not the same nanosecond 460 times. `statx` works there, and the file clock is coarse: a directory made and filled within one tick has both times equal. -- **The nanosecond rule did not fix it.** The row failed 3 times in 150 under the millisecond rule, 5 times in 300 and 8 times in 300 under the nanosecond rule. - -A positive birth time is now kept as it is, by the CLI and the hook alike, and the change time is not read. - -| Rule | Mutation | Red | -| --- | --- | --- | -| a birth equal to the change time is kept, CLI | `\|\| stat.birthtimeNs === stat.ctimeNs` put back in `identityFromStat` | 2 in the unit test (the equal case, and the identity of a stat carrying no ns fields), 1 in the fixture test (`linux-birth-equal-to-change-time-is-a-birth`) | -| the same, hook | the same in `consent.cjs` | 1 in the hook test "a directory's identity follows the fixture" | -| an identity does not move when an entry is added | the same, CLI, run on Linux | "keeps its identity once an entry is added", 5 runs out of 5. It cannot fail on macOS, where APFS never had the two times equal over 500 directories | -| the row "copy at the path", overlay on btrfs, root | the same, both sides | 8 runs failed out of 300. With the rule removed: 0 out of 400 | -| the same, on an ext4 loop device, as the unprivileged `node` user | the same, both sides | 81 runs failed out of 200. With the rule removed: 0 out of 400 | - -On that ext4 device, as `node`, the whole consent table and the reader test passed 50 runs out of 50 with the rule removed. A fresh directory there had its birth equal to its change time; after an entry was added 50 ms later, the change time had moved and the birth had not. - -### Where Linux refuses `statx` - -Simulated on the same ext4 device: `strace -f -e inject=statx:error=ENOSYS` makes every `statx` fail as on a kernel before 4.11, and libuv falls back to `stat`. A fresh directory then had its birth equal to its change time, and after an entry was added both had moved together. The consent table, its assertions made soft so that each row reports what it finally stored, 36 rows, run as root: - -| Rule | Rows as expected | Stored less | Stored more | -| --- | --- | --- | --- | -| a birth equal to the change time is kept (this change), 3 runs | 1 | 34 | 1, every run: "on, off, the clone moved, on" stored `a2`, made after `off`, and not `a3` | -| a birth equal to the change time is no birth (the rule before), 2 runs | 33 | 0 | 3, every run: "unmounted, made again" and "copy at the path", the residual where there is no birth time, and "unstattable", which `chmod` cannot shut to root | - -With the birth taken from the change time, the identity moves at each entry added to the git dir. Most rows then store less than they should. The one that stores more does so because `off` closed only the interval of the identity it read, which was no longer the one `on` opened. - -`off` now also closes the open interval its key names when that interval was opened at the same path. A copy elsewhere carries the token but not the path, and is left alone. The same simulation, as the unprivileged `node` user: - -| Rule | Rows as expected | Stored less | Stored more | -| --- | --- | --- | --- | -| `off` closes by identity, and by the key's token at its path, 3 runs | 1 | 35 | 0 | -| the same, `statx` allowed, 2 runs | 36 | 0 | 0 | - -| Rule | Mutation | Red | -| --- | --- | --- | -| `off` closes the interval its key names at its path | the token clause removed | 1, "closes the interval its key names at its path, though the identity read there moved since `on`" | -| only at its path | the path condition removed | 1, "leaves the interval of the clone a copy was made from, its key naming it at another path" | - -A residual limit of the usage contract: the clone stops being measured, and it fails closed. - -### The other guards of these commits - -| Guard | Mutation | Red | -| --- | --- | --- | -| a directory is not read as a transcript | `if (!stats.isFile()) throw` removed | 1, "fails on a transcript it cannot read". The test now asks for `/is not a file/`: before, it passed on POSIX without the guard, because reading a directory throws `EISDIR` there anyway | -| the husky hook is named with `/` | not mutated: the path is the same on POSIX. CI run 38088839814, before the fix, failed 2 tests on Windows with `".husky\\prepare-commit-msg"` | Windows only | - -### The session id join, probed against the real tool - -`scripts/probe-identifier-join.cjs` runs the installed `claude -p` against a dead API, with a fake key and an empty home, and judges what one `SessionStart` hook and the transcript left, through the shipped `isClaudePayload` and `personIsPresent`. On 2.1.295 it exits 0 after its 90 s bound: `the join holds`. The CI job `cli / Identifier join (Claude Code)` runs it again. Its first CI run, on Ubuntu with 2.1.296, failed a first version that required `CLAUDE_PID` to be the hook's parent: `CLAUDE_PID 2565 is not the hook's parent 2588`. A shell stands between Claude and its hooks there, so the probe now requires `CLAUDE_PID` to be set, the variable the hook reads first, and the hook's comment no longer claims its parent is Claude. - -| Check | Mutation | Red | -| --- | --- | --- | -| a hook fired before the API call | the check made `false` | 2, "is a change of the tool when no hook fired before the API call" and "exits 1 when the tool fires no hook" | -| the Claude-only guard accepts Claude's payload | the same | 1, "is a change when the hook's session variable is not its payload's id" | -| the transcript is under the config dir's `projects` | the same | 1, "is a change when the transcript is not under the config dir's projects" | -| a transcript was written | the same | 1, "is a change when no transcript was written" | -| every transcript line names the hook's session | the same | 2, "is a change when a transcript line names another session" and "is a change when no transcript line names a session" | -| at least one line names it | `ids.length === 0` removed | 1, "is a change when no transcript line names a session" | -| `CLAUDE_PID` is set, so the hook does not fall back to its parent pid | the same | 1, "is a change when CLAUDE_PID is gone, leaving the hook's parent pid to name Claude" | -| a headless run is not a person present | the same | 1, "is a change when a headless run reads as a person at the keyboard" | - -### Presence, observed on each surface - -Observed with the maintainer on 2026-10-12, recorded in [target.md](target.md): the terminal (`cli`) and the desktop app (`claude-desktop`) set `ATTENDED=1` and show a block's reason; the VS Code extension (`claude-vscode`) sets no `ATTENDED` and shows neither a reason nor a `systemMessage`. The rule was "any entrypoint not starting with `sdk`", which would have blocked a surface never seen; it is now the two surfaces seen to show the reason. - -| Rule | Mutation | Red | -| --- | --- | --- | -| the desktop app asks | `claude-desktop` removed from the set | row "the desktop app" of "presence: only ATTENDED=1 from a surface seen to show a block asks" | -| a surface never seen does not | the set replaced by the previous "not `sdk`" rule | rows "attended, from a surface never seen" and "entrypoint empty" | -| `ATTENDED=1` is required | the `ATTENDED` condition removed | rows "unattended", "attended variable absent" and "attended is true, not 1", and the test "presence: a typed declaration is not run for nobody" | -| the VS Code extension is not asked | `claude-vscode` added to the set and the `ATTENDED` condition removed | row "the VS Code extension", with the three rows above. It holds by two independent conditions, so no single mutation reaches it: adding `claude-vscode` alone leaves it green, since the extension sets no `ATTENDED` | - -The matrix judges every row before it fails, so a mutation names all the rows it turns red. - -### Transcripts no consent can cover are not read - -A first ingest on a real 6.2 GB Claude history never finished in 22 minutes. With transcripts last written more than a day before the first interval left unread: 246 s, then 1.1 s. On a real session in a sandboxed telemetry directory, the stored calls match an independent count of the same transcripts: 5 calls, 217,337 tokens, per session alike. The Docker consent table is unchanged: 36 of 36 with `statx`, 2 runs; with it refused, 0 rows storing more, 3 runs. - -| Rule | Mutation | Red | -| --- | --- | --- | -| a day of margin for the file clock | the margin removed | "reads a transcript whose file clock lags the call clock, within a day" | -| skipping stores what reading everything stores | the margin added instead of taken | the above, and "stores exactly what reading every transcript stores" | diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/prototype/check.py b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/prototype/check.py deleted file mode 100644 index ab3534f2b..000000000 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/prototype/check.py +++ /dev/null @@ -1,82 +0,0 @@ -"""Prototype: two tool readers -> one usage-record contract. Asserts on synthetic fixtures.""" -import json, glob, os, sys - -HERE = os.path.dirname(os.path.abspath(__file__)) -FX = os.path.join(HERE, "fixtures") - -def rec(tool, key, session, agent, at, model, inp, out, cr, cw, cw1h, reasoning, cwd, branch, version): - return dict(tool=tool, key=key, session_id=session, agent=agent, at=at, model=model, - input=inp, output=out, cache_read=cr, cache_write=cw, cache_write_1h=cw1h, - reasoning=reasoning, cwd=cwd, git_branch=branch, tool_version=version) - -def claude(files): - best = {} - for f in files: - for raw in open(f): - r = json.loads(raw); m = r.get("message") or {}; u = m.get("usage") - if r.get("type") != "assistant" or not isinstance(u, dict) or m.get("model") == "": - continue - key = f"{m['id']}:{r['requestId']}" if r.get("requestId") else f"{m['id']}:{r['sessionId']}:{r['timestamp']}" - total = sum(u.get(k) or 0 for k in ("input_tokens", "output_tokens", "cache_read_input_tokens", "cache_creation_input_tokens")) - if key not in best or total > best[key][0]: - best[key] = (total, r) - out = [] - for key, (_, r) in best.items(): - m = r["message"]; u = m["usage"]; its = u.get("iterations") or [] - msgs = [i for i in its if i.get("type") == "message"] or [u] - cw = sum(i.get("cache_creation_input_tokens") or 0 for i in msgs) - splits = [i.get("cache_creation") for i in msgs] - cw1h = sum(s.get("ephemeral_1h_input_tokens", 0) for s in splits) if all(isinstance(s, dict) for s in splits) else None - agent = "subagent" if r.get("isSidechain") or r.get("agentId") else "main" - base = (r["sessionId"], None, r["timestamp"]) - out.append(rec("claude-code", key, r["sessionId"], agent, r["timestamp"], m["model"], - u.get("input_tokens"), u.get("output_tokens"), u.get("cache_read_input_tokens"), - cw, cw1h, None, r.get("cwd"), r.get("gitBranch"), r.get("version"))) - for n, a in enumerate(i for i in its if i.get("type") == "advisor_message"): - out.append(rec("claude-code", f"{key}#advisor{n}", r["sessionId"], "advisor", r["timestamp"], a.get("model"), - a.get("input_tokens"), a.get("output_tokens"), a.get("cache_read_input_tokens"), - a.get("cache_creation_input_tokens"), None, None, r.get("cwd"), r.get("gitBranch"), r.get("version"))) - return out - -def codex(path): - out = []; session = version = cwd = branch = model = None; last_total = None - for raw in open(path): - r = json.loads(raw); p = r.get("payload") or {} - if r["type"] == "session_meta": - session, version, cwd = p["id"], p.get("cli_version"), p.get("cwd") - branch = (p.get("git") or {}).get("branch") - elif r["type"] == "turn_context": - model, cwd = p.get("model"), p.get("cwd", cwd) - elif p.get("type") == "token_count" and p.get("info"): - total = p["info"]["total_token_usage"]["total_tokens"] - if total == last_total: # verbatim re-emission, not a new call - continue - last_total = total; l = p["info"]["last_token_usage"] - out.append(rec("codex", f"{session}:{total}", session, "main", r["timestamp"], model, - l["input_tokens"] - l["cached_input_tokens"], l["output_tokens"], l["cached_input_tokens"], - l.get("cache_write_input_tokens"), None, l.get("reasoning_output_tokens"), cwd, branch, version)) - return out - -c = claude(sorted(glob.glob(FX + "/claude-*.jsonl")) + glob.glob(FX + "/subagents/*.jsonl")) -x = codex(FX + "/codex-rollout.jsonl") -by = {r["key"]: r for r in c + x} -expect = { - "msg_A:req_A": dict(agent="main", input=3, output=250, cache_read=1000, cache_write=200, cache_write_1h=200), - "msg_B:req_B": dict(agent="main", input=4, output=300, cache_read=5000, cache_write=2283, cache_write_1h=2283), - "msg_B:req_B#advisor0": dict(agent="advisor", model="claude-fable-5-1", input=90000, output=700, cache_write_1h=None), - "msg_C:req_C": dict(agent="subagent", model="claude-sonnet-5-5", input=5, cache_write=60, cache_write_1h=0), - "cx-1:1050": dict(input=200, cache_read=800, output=50, reasoning=20, cache_write_1h=None), - "cx-1:2630": dict(input=500, cache_read=1000, output=80, reasoning=20, model="gpt-5.5-codex"), -} -fail = 0 -keys = [r["key"] for r in c + x] -if len(keys) != len(set(keys)): - print("DUPLICATE KEYS"); fail += 1 -if set(by) != set(expect): - print("KEYS", sorted(set(by) ^ set(expect))); fail += 1 -for k, e in expect.items(): - for f, v in e.items(): - if by.get(k, {}).get(f, "MISSING") != v: - print(f"FAIL {k}.{f} = {by.get(k, {}).get(f, 'MISSING')} expected {v}"); fail += 1 -print(f"{len(c)} claude + {len(x)} codex records, {fail} failures") -sys.exit(1 if fail else 0) diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/prototype/fixtures/claude-main.jsonl b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/prototype/fixtures/claude-main.jsonl deleted file mode 100644 index 69c2903d4..000000000 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/prototype/fixtures/claude-main.jsonl +++ /dev/null @@ -1,4 +0,0 @@ -{"type":"assistant","sessionId":"s-1","requestId":"req_A","timestamp":"2026-10-07T10:00:00.000Z","version":"2.1.292","cwd":"/work/repo-wt-a","gitBranch":"feat/a","message":{"id":"msg_A","model":"claude-opus-5-5","usage":{"input_tokens":3,"output_tokens":10,"cache_read_input_tokens":1000,"cache_creation_input_tokens":200,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":200}}}} -{"type":"assistant","sessionId":"s-1","requestId":"req_A","timestamp":"2026-10-07T10:00:01.000Z","version":"2.1.292","cwd":"/work/repo-wt-a","gitBranch":"feat/a","message":{"id":"msg_A","model":"claude-opus-5-5","usage":{"input_tokens":3,"output_tokens":250,"cache_read_input_tokens":1000,"cache_creation_input_tokens":200,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":200}}}} -{"type":"assistant","sessionId":"s-1","requestId":"req_B","timestamp":"2026-10-07T10:01:00.000Z","version":"2.1.292","cwd":"/work/repo-wt-a/src","gitBranch":"feat/a","message":{"id":"msg_B","model":"claude-opus-5-5","usage":{"input_tokens":4,"output_tokens":300,"cache_read_input_tokens":5000,"cache_creation_input_tokens":2283,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":1436},"iterations":[{"type":"message","input_tokens":2,"output_tokens":100,"cache_read_input_tokens":2500,"cache_creation_input_tokens":1436,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":1436}},{"type":"advisor_message","model":"claude-fable-5-1","input_tokens":90000,"output_tokens":700,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},{"type":"message","input_tokens":2,"output_tokens":200,"cache_read_input_tokens":2500,"cache_creation_input_tokens":847,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":847}}]}}} -{"type":"assistant","sessionId":"s-1","requestId":"req_S","timestamp":"2026-10-07T10:02:00.000Z","version":"2.1.292","cwd":"/work/repo-wt-a","gitBranch":"feat/a","message":{"id":"msg_S","model":"","usage":{"input_tokens":0,"output_tokens":0}}} diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/prototype/fixtures/claude-resumed-copy.jsonl b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/prototype/fixtures/claude-resumed-copy.jsonl deleted file mode 100644 index b13b25955..000000000 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/prototype/fixtures/claude-resumed-copy.jsonl +++ /dev/null @@ -1 +0,0 @@ -{"type":"assistant","sessionId":"s-2","requestId":"req_A","timestamp":"2026-10-07T10:00:01.000Z","version":"2.1.292","cwd":"/work/repo-wt-a","gitBranch":"feat/a","message":{"id":"msg_A","model":"claude-opus-5-5","usage":{"input_tokens":3,"output_tokens":10,"cache_read_input_tokens":1000,"cache_creation_input_tokens":200,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":200}}}} diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/prototype/fixtures/codex-rollout.jsonl b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/prototype/fixtures/codex-rollout.jsonl deleted file mode 100644 index 3e93a17d6..000000000 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/prototype/fixtures/codex-rollout.jsonl +++ /dev/null @@ -1,6 +0,0 @@ -{"type":"session_meta","timestamp":"2026-10-07T09:00:00.000Z","payload":{"id":"cx-1","cwd":"/work/repo-wt-b","cli_version":"0.150.0","git":{"branch":"fix/b"}}} -{"type":"turn_context","timestamp":"2026-10-07T09:00:01.000Z","payload":{"turn_id":"t-1","model":"gpt-5.5-codex","cwd":"/work/repo-wt-b"}} -{"type":"event_msg","timestamp":"2026-10-07T09:00:05.000Z","payload":{"type":"token_count","info":{"total_token_usage":{"input_tokens":1000,"cached_input_tokens":800,"cache_write_input_tokens":0,"output_tokens":50,"reasoning_output_tokens":20,"total_tokens":1050},"last_token_usage":{"input_tokens":1000,"cached_input_tokens":800,"cache_write_input_tokens":0,"output_tokens":50,"reasoning_output_tokens":20,"total_tokens":1050}}}} -{"type":"event_msg","timestamp":"2026-10-07T09:00:09.000Z","payload":{"type":"token_count","info":{"total_token_usage":{"input_tokens":2500,"cached_input_tokens":1800,"cache_write_input_tokens":0,"output_tokens":130,"reasoning_output_tokens":40,"total_tokens":2630},"last_token_usage":{"input_tokens":1500,"cached_input_tokens":1000,"cache_write_input_tokens":0,"output_tokens":80,"reasoning_output_tokens":20,"total_tokens":1580}}}} -{"type":"event_msg","timestamp":"2026-10-07T09:00:09.500Z","payload":{"type":"token_count","info":{"total_token_usage":{"input_tokens":2500,"cached_input_tokens":1800,"cache_write_input_tokens":0,"output_tokens":130,"reasoning_output_tokens":40,"total_tokens":2630},"last_token_usage":{"input_tokens":1500,"cached_input_tokens":1000,"cache_write_input_tokens":0,"output_tokens":80,"reasoning_output_tokens":20,"total_tokens":1580}}}} -{"type":"event_msg","timestamp":"2026-10-07T09:00:10.000Z","payload":{"type":"token_count","info":null}} diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/prototype/fixtures/subagents/agent-x.jsonl b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/prototype/fixtures/subagents/agent-x.jsonl deleted file mode 100644 index 86bd05984..000000000 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/prototype/fixtures/subagents/agent-x.jsonl +++ /dev/null @@ -1 +0,0 @@ -{"type":"assistant","sessionId":"s-1","agentId":"x","isSidechain":true,"requestId":"req_C","timestamp":"2026-10-07T10:03:00.000Z","version":"2.1.292","cwd":"/work/repo-wt-a","gitBranch":"feat/a","message":{"id":"msg_C","model":"claude-sonnet-5-5","usage":{"input_tokens":5,"output_tokens":40,"cache_read_input_tokens":300,"cache_creation_input_tokens":60,"cache_creation":{"ephemeral_5m_input_tokens":60,"ephemeral_1h_input_tokens":0}}}} diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/real-history-oracle.md b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/real-history-oracle.md index f3f288b83..bda1833af 100644 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/real-history-oracle.md +++ b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/real-history-oracle.md @@ -14,8 +14,8 @@ path or token volume of the person whose history it was. 2. **CLI.** The built `cli/dist/cli.js` ran `aidd telemetry report --axis day --json` from that repository. `HOME`, `CLAUDE_CONFIG_DIR`, `AIDD_TELEMETRY_DIR`, `AIDD_USER_CONFIG_DIR` and `XDG_CONFIG_HOME` all pointed into the sandbox, and `CLAUDE_CODE_SESSION_ID` was unset. -3. **Independent count.** The `claude()` reader of [`prototype/check.py`](./prototype/check.py) - (Python, written before the CLI) read the same files. Its records were grouped by the UTC day +3. **Independent count.** The `claude()` reader of `prototype/check.py` + (Python, written before the CLI, removed after the merge; git history keeps it) read the same files. Its records were grouped by the UTC day of `at`, and each day's four counters and record count were compared with the CLI's row. > **Note, 2026-10-10.** Step 1 describes the opt-in of the day the run was recorded (`b2b732eb`, diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/target.md b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/target.md index 14545d35c..94e552d0c 100644 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/target.md +++ b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/target.md @@ -102,5 +102,5 @@ Root-scoped bindings and their adoption/`ambiguous` rule; cross-machine binding ## Oracles -Fixture + mutation per rule (contract/check.py: 6/6 killed); 16 counting pitfalls; ccusage +Fixture + mutation per rule (the prototype `check.py`, since removed: 6/6 killed); 16 counting pitfalls; ccusage diff; OTel `api_request` by `request_id`; Analytics API daily totals. diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/usage-record-contract.md b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/usage-record-contract.md deleted file mode 100644 index 184be68b6..000000000 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/context/usage-record-contract.md +++ /dev/null @@ -1,45 +0,0 @@ -# Usage record contract (draft v0) - -One record per billed model call. Facts only: no cost, person, task, ticket or step. -Those are derived downstream, once, for every tool. - -| Field | Type | Meaning | -| --- | --- | --- | -| `tool` | string | `claude-code`, `codex`, … | -| `tool_version` | string \| null | version that wrote the source line | -| `key` | string | stable across re-reads, unique within `tool`; storage is idempotent on `(tool, key)` | -| `session_id` | string | the tool's own session id | -| `agent` | `main` \| `subagent` \| `advisor` | who made the call | -| `at` | ISO-8601 UTC | when the call was recorded | -| `model` | string \| null | model that billed this call | -| `input` | int \| null | input tokens **excluding** any cache read or write | -| `output` | int \| null | output tokens **including** reasoning | -| `cache_read` | int \| null | input tokens served from cache | -| `cache_write` | int \| null | input tokens written to cache, all TTLs | -| `cache_write_1h` | int \| null | of which 1-hour TTL; `null` when the tool does not say | -| `reasoning` | int \| null | of which reasoning, a subset of `output`; `null` when the tool does not say | -| `cwd` | string \| null | raw working directory of that line, joined to a worktree downstream; local only | -| `git_branch` | string \| null | raw, as the line says it; the reader does not interpret it. Attribution uses it only to find the branch's declaration snapshot, together with `repository_id` | - -Rules: an unknown value is `null`, never `0`. A shape the reader does not recognise is reported, never guessed. - -## Per-tool normalisation (proved on fixtures, each by a mutation) - -Claude Code -- key `message.id:requestId`; requestless lines `message.id:sessionId:timestamp`; keep the row with the largest total (streaming snapshots, resumed copies). -- drop ``. -- `cache_write` and `cache_write_1h` summed over `iterations[type=message]`; the top-level `cache_creation` split reflects only the first iteration (measured, 86 requests on 2026-10-07; ccusage 20.0.26 under-counts by it). -- each `iterations[type=advisor_message]` is its own record, `agent: advisor`, its own model; top-level usage excludes it. -- `isSidechain` or `agentId` → `subagent`. - -Codex -- key `session_meta.id:total_token_usage.total_tokens` (monotonic: 3,765/3,765 on 50 real rollouts). -- skip a `token_count` whose cumulative total did not move (verbatim re-emission, 3.5 % of events on 200 rollouts). -- `input = last.input_tokens − last.cached_input_tokens` (OpenAI input is inclusive of cache). -- model from the preceding `turn_context`. - -## Open - -- Codex forks and sub-agents: `session_meta` carries `forked_from_id`, `parent_thread_id`, `subagent_history_start_ordinal`. Does a forked rollout replay parent `token_count` events? If so the key above double-counts across sessions. -- Claude `cache_write_1h` across iterations when a later iteration writes 1h: fixture asserts the sum; real occurrence not yet measured. -- `cwd` per line for Codex: `turn_context.cwd` assumed to follow directory changes; not measured. diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-1.md b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-1.md deleted file mode 100644 index 0f5ea1c98..000000000 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-1.md +++ /dev/null @@ -1,93 +0,0 @@ ---- -status: implemented ---- - - - -# Instruction: Remove the previous telemetry from the CLI - -## Architecture projection - -> Tree of the final files. ✅ create · ✏️ modify · ❌ delete - -```txt -cli/ -├── src/ -│ ├── cli.ts ✏️ no telemetry registration (re-added in phase 4) -│ ├── contexts/telemetry/ ❌ whole directory -│ ├── contexts/tools/domain/contracts.ts ✏️ drop telemetryLocalRead, telemetryJournalHost, telemetryTaskAttributable -│ ├── contexts/tools/domain/registry.ts ✏️ drop journalHostToAiToolId -│ ├── contexts/tools/domain/profiles/*/profile.ts ✏️ drop the three telemetry fields -│ ├── contexts/tools/domain/profiles/claude/claude-transcript-location.ts ❌ -│ ├── contexts/tools/domain/profiles/codex/codex-transcript-location.ts ❌ -│ ├── kernel/measurement.ts ❌ -│ ├── kernel/errors.ts ✏️ drop telemetry-only errors (keep any still thrown elsewhere) -│ ├── kernel/paths.ts ✏️ drop RUNS_ENTRY/resolvedRunsDir unless framework still needs them -│ ├── kernel/reading/home-dir.ts ✏️ drop resolveAiddConfigDir (only caller deleted) -│ ├── runtime/git/git-adapter.ts ✏️ drop the commit-trailer delegate methods -│ ├── runtime/wiring/telemetry.ts ❌ -│ ├── runtime/wiring/framework.ts ✏️ no TelemetryDeps -│ ├── runtime/wiring/installed-plugins-from-manifest.ts ❌ if telemetry was its only caller -│ ├── presentation/commands/telemetry.ts ❌ -│ └── presentation/display/{telemetry-*,cost-report-*}.ts ❌ -├── tests/ ❌ every test of the above; ✏️ architecture allow-lists -├── biome.json ✏️ drop telemetry layer overrides -├── mutation-scopes.json, package.json ✏️ drop the telemetry mutation scope and script -├── scripts/check-bundle-size.mjs ✏️ untouched here: the budget is set once, in phase 9 -├── scripts/smoke-tools.sh ✏️ drop telemetry smoke steps -└── tests/golden/snapshots/help/surface.json ✏️ regenerated, no telemetry -``` - -## User Journey - -```mermaid -flowchart TD - A[Developer runs aidd --help] --> B[No telemetry command listed] - B --> C[Every other command behaves as before] -``` - -## Test Scope - -```mermaid ---- -title: Test scope ---- -journey - section Setup - Build the CLI from the branch => binary built: 5: system - section Happy path - Run aidd --help => no telemetry entry, every other command listed as before: 5: cli - Run the full CLI suite => green with no telemetry test left: 5: cli - section Edge case - straggler - A file still imports a deleted module => typecheck and knip fail => the straggler is named: 1: cli -``` - -## Tasks to do - -### `1)` Delete the context and its wiring - -> Nothing under `contexts/telemetry` survives, and nothing imports it. - -1. Delete `cli/src/contexts/telemetry/`, `runtime/wiring/telemetry.ts`, `presentation/commands/telemetry.ts`, and the telemetry and cost-report display modules. -2. Remove the registration from `cli.ts` and `TelemetryDeps` from `runtime/wiring/framework.ts`. -3. Remove telemetry-only code living elsewhere: `kernel/measurement.ts`, the three profile fields and their contract, the transcript locations, `journalHostToAiToolId`, `resolveAiddConfigDir`, telemetry-only errors, the git adapter's trailer delegate methods. -4. Keep `kernel/paths.ts` `AIDD_DIR`/`AIDD_CONFIG_FILENAME`; keep `RUNS_ENTRY` only if `framework` still reads it, otherwise delete it and its gitignore entry producer. - -### `2)` Delete the tests and update the guards - -> The suites and architecture guards describe the CLI that remains. - -1. Delete every telemetry test, fixture and in-memory port double listed by `grep -ril telemetry cli/tests`. -2. Remove `telemetry` from `ALLOWED`, `PUBLIC_MODULES`, folder-size, orchestrator-deps, tool-addition-cost, catches-that-swallow and self-reentry baselines; drop the biome telemetry overrides. -3. Remove the telemetry mutation scope and script; regenerate the help golden with `UPDATE_HELP_GOLDEN=1`. -4. Drop telemetry steps from `scripts/smoke-tools.sh`. Leave the bundle budget alone. -5. Before deleting, copy into phase 8's notes every location the previous version wrote: the sink dir resolution (`AIDD_TELEMETRY_DIR`, `AIDD_USER_CONFIG_DIR`, `%APPDATA%\aidd\telemetry`, `~/.config/aidd/telemetry`), `identity.json` (`%APPDATA%\aidd` or `~/.config/aidd`), the trailer delegate name and hook line, `aidd_docs/runs` and `AIDD_RUNS_DIR`. - -## Test acceptance criteria - -| Task | Acceptance criteria | -| ---- | ------------------- | -| 1 | `grep -ril telemetry cli/src` finds nothing but comments explaining the `.aidd/config.json` file `clean` keeps | -| 1 | `aidd --help` lists no `telemetry` command; `setup`, `plugin`, `clean`, `framework build` behave as before | -| 2 | typecheck, lint, arch, knip, jscpd, the three tiers and the build with its budget are green | -| 2 | No architecture baseline names a context or file that no longer exists | diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-2.md b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-2.md deleted file mode 100644 index 2f0932730..000000000 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-2.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -status: implemented ---- - - - -# Instruction: Remove the previous telemetry from the plugins, scripts, CI and docs - -## Architecture projection - -> Tree of the final files. ✅ create · ✏️ modify · ❌ delete - -```txt -. -├── plugins/aidd-telemetry/ -│ ├── hooks/ ❌ journal.cjs, lib/, opencode-plugin.js, hooks.json (rebuilt in phase 6) -│ ├── skills/{00-init,01-cost,02-check}/ ❌ (rebuilt in phase 9) -│ ├── README.md ✏️ "being rebuilt" until phase 9 -│ └── CATALOG.md ✏️ regenerated -├── plugins/aidd-pm/skills/04-spec/ -│ ├── actions/01-build.md ✏️ no Declare step, no backlog-link test row -│ ├── references/backlog-link.md ❌ -│ └── assets/backlog-link-template.json ❌ -├── plugins/aidd-dev/skills/01-plan/ -│ ├── actions/04-plan.md ✏️ no backlog-link step or test -│ └── SKILL.md ✏️ no step-end marker -├── plugins/aidd-orchestrator/skills/ -│ ├── 01-sdlc/SKILL.md ✏️ no "Say when this orchestration is over" -│ ├── 01-sdlc/references/01-frame.md ✏️ no backlog-link paragraph -│ ├── 00-async-dev/SKILL.md ✏️ no step-end marker -│ └── 02-backlog/SKILL.md ✏️ no step-end marker -├── scripts/__tests__/aidd-telemetry-*.test.js, telemetry-*.test.js, a-backlog-link-*.test.js ❌ -├── scripts/__tests__/fixtures/ (telemetry-only fixtures) ❌ -├── scripts/lib/telemetry-reference-week.cjs, scripts/probe-identifier-join.cjs ❌ -├── lefthook.yml ✏️ no prepare-commit-msg trailer job, no prompts-doc job if it only served telemetry -├── .github/workflows/cli-ci.yml, ci.yml ✏️ no journal replay, no telemetry check, no telemetry e2e exclusion -├── aidd_docs/product/{cost-report,metrics}-contract.md ❌ -├── aidd_docs/runs/ ❌ -├── aidd_docs/tasks/2026_09/{the-upward-link,flow-and-versions,install-surfaces-agree, fewer-verbs,one-place-per-nature,telemetry-at-scale} ✏️ status: superseded -├── cli/aidd_docs/memory/telemetry.md, cli/.claude/skills/telemetry/ ❌ -└── memory, README, docs/*.md, cli/README.md, cli/ARCHITECTURE.md ✏️ no claim about the previous telemetry -``` - -## User Journey - -```mermaid -flowchart TD - A[Contributor commits on the repo] --> B[No trailer job runs, commit succeeds] - C[User runs the SDLC or spec or plan skill] --> D[No backlog-link.json written, no step-end echo] -``` - -## Test Scope - -```mermaid ---- -title: Test scope ---- -journey - section Setup - Check out the branch with hooks installed => lefthook ready: 5: system - section Happy path - Run the repository scripts suite => green with no telemetry test left: 5: cli - Run pre-commit on a commit => every job green, no trailer job: 5: cli - section Edge case - dangling reference - A doc or skill still names a deleted file => markdown links and comments-name-files-that-exist fail: 1: cli -``` - -## Tasks to do - -### `1)` Empty the plugin and unhook the other skills - -> No plugin emits or writes anything for the previous telemetry. - -1. Delete the aidd-telemetry hooks and skills; leave `plugin.json` and a README saying the plugin is being rebuilt. -2. Remove the backlog-link step, reference, template and test rows from `aidd-pm:04-spec` and `aidd-dev:01-plan`, and the paragraph from `01-sdlc/references/01-frame.md`. -3. Remove every `aidd:step-end` echo and the SDLC's "Say when this orchestration is over" section. - -### `2)` Scripts, hooks, CI - -> The repository's own gates no longer mention the previous telemetry. - -1. Delete the telemetry script tests, their fixtures and the two telemetry scripts. -2. Remove the `prepare-commit-msg` trailer job from `lefthook.yml`, and the prompts-doc job if telemetry was its only input. -3. Remove the journal replay, `telemetry check` and telemetry e2e exclusion from the workflows. - -### `3)` Docs and task history - -> Docs describe what exists; past work stays readable as history. - -1. Delete the two product contracts, `aidd_docs/runs/`, the CLI telemetry memory and skill. -2. Rewrite each memory, README and docs line that claims the previous telemetry (list in the explore notes) to say nothing or point at phase 9's docs. -3. Mark the pending telemetry task folders `status: superseded`; leave done folders untouched. - -## Test acceptance criteria - -| Task | Acceptance criteria | -| ---- | ------------------- | -| 1 | `grep -rn "backlog-link\|aidd:step-end" plugins` finds nothing | -| 2 | `node --test 'scripts/__tests__/**/*.test.js'` and `lefthook run pre-commit` are green | -| 2 | A commit in a clone that has a stale `aidd-session-trailer.sh` delegate, and one that has none, both succeed | -| 3 | markdown-links, referenced-paths, comments-name-files-that-exist and the CATALOG/README count checks are green | -| 3 | kanban shows no pending telemetry task from before this plan | diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-3.md b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-3.md deleted file mode 100644 index f07a73e42..000000000 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-3.md +++ /dev/null @@ -1,93 +0,0 @@ ---- -status: implemented ---- - - - -# Instruction: Count, Claude transcripts to usage records - -## Architecture projection - -> Tree of the final files. ✅ create · ✏️ modify · ❌ delete - -```txt -cli/ -├── src/contexts/telemetry/ -│ ├── domain/usage-record.ts ✅ the common record (context/usage-record-contract.md) -│ ├── domain/formats/claude-transcript-usage.ts ✅ one transcript line to candidate records, pure -│ ├── domain/usage-fold.ts ✅ global fold: one record per key, deterministic winner -│ ├── domain/ports/transcript-source.ts ✅ list files, read bytes from an offset -│ └── infrastructure/claude-transcript-source-adapter.ts ✅ recursive walk of /projects -└── tests/ - ├── fixtures/claude-usage/ ✅ ported from context/prototype/fixtures, synthetic ids only - ├── contexts/telemetry/domain/*.unit.test.ts ✅ one test per rule, each with its mutation noted - └── contexts/telemetry/infrastructure/claude-transcript-source-adapter.integration.test.ts ✅ -``` - -## User Journey - -```mermaid -flowchart TD - A[Transcripts: main, resumed copy, subagent, set-aside] --> B[Per-line candidates] - B --> C[Global fold by key] - C --> D[One usage record per billed call, advisor calls apart] -``` - -## Test Scope - -```mermaid ---- -title: Test scope ---- -journey - section Setup - Copy the claude-usage fixtures to a temp config dir => transcripts in place: 5: system - section Happy path - Read and fold all transcripts => exactly the expected records, field for field: 5: system - Read the same files again, in reverse order => identical records: 5: system - section Edge case - streaming and resumed copies - Same key in three lines with growing totals => fold => the largest total wins: 1: system - section Edge case - advisor iterations - A message with advisor iterations => fold => one main record plus one advisor record per iteration: 1: system - section Edge case - cache writes across iterations - Later iteration writes 1h cache => fold => cache_write and cache_write_1h are iteration sums: 1: system - section Edge case - unknown shape - A usage line missing a counter => read => record field unknown and the shape reported, never zero: 1: system - section Edge case - unterminated last line - File ends mid-line => read => the partial line is not consumed: 1: system -``` - -## Tasks to do - -### `1)` The record and the per-line rules - -> A pure function turns one transcript line into zero or more candidate records. - -1. Define `UsageRecord` exactly as `context/usage-record-contract.md`, `tool: "claude-code"`; unknown is `null`. -2. Key `message.id:requestId`, else `message.id:sessionId:timestamp`; drop ``; `agent` is `subagent` when `isSidechain` or `agentId`. -3. `cache_write` and `cache_write_1h` summed over `iterations[type=message]`, top-level when there are none; `cache_write_1h` unknown when any split is missing. -4. Each `iterations[type=advisor_message]` is its own record, key `#advisor`, its own model. -5. A recognisable usage line with an unexpected shape yields an "unrecognised" outcome counted and surfaced, never a guessed record. - -### `2)` The global fold - -> Order-independent: one winner per key. - -1. Winner is the largest total; tie, the earliest `at`; tie, the lowest `session_id`. -2. Fold across every file, not per file. - -### `3)` The transcript source - -> Every transcript under the projects root, read from a byte offset. - -1. Walk `/projects` recursively, including `subagents/` and `*.jsonl.superseded-*` set-asides. -2. Read from a given byte offset; never consume an unterminated last line; report file size and identity (device+inode or equivalent) so a shrunk or replaced file is re-read whole. - -## Test acceptance criteria - -| Task | Acceptance criteria | -| ---- | ------------------- | -| 1 | Each rule has a unit test that fails when that rule alone is mutated (iteration sum, advisor key, synthetic filter, sidechain agent, requestless key) | -| 2 | Reading files in any order gives the same records; mutating the tie-break or folding per file turns a named test red | -| 3 | Subagent and set-aside files are found; a file appended mid-line yields its complete lines only; a truncated file is re-read whole | -| all | No fixture holds a real session id, prompt, path or name | diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-4.md b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-4.md deleted file mode 100644 index 2cf7accdd..000000000 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-4.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -status: implemented ---- - - - -# Instruction: Store, the ledger, ingest and consent - -## Architecture projection - -> Tree of the final files. ✅ create · ✏️ modify · ❌ delete - -```txt -cli/ -├── src/ -│ ├── cli.ts ✏️ register the telemetry group -│ ├── presentation/commands/telemetry.ts ✅ group + `ingest` (hidden from the skill surface, used by the hook) -│ ├── runtime/wiring/telemetry.ts ✅ -│ ├── runtime/git/git-adapter.ts ✏️ rev-parse roots, origin url, root commit, config get/set/unset, branch reflog creation -│ └── contexts/telemetry/ -│ ├── domain/repository-identity.ts ✅ hash(host/owner/repo) from origin, else root commit -│ ├── domain/telemetry-consent.ts ✅ .aidd/config.json telemetry.enabled + AIDD_TELEMETRY=0 -│ ├── domain/ports/{usage-ledger,repository-locator,binding-snapshot-store}.ts ✅ -│ ├── application/ingest-usage-use-case.ts ✅ -│ └── infrastructure/{usage-ledger-adapter,repository-locator-adapter,binding-snapshot-store-adapter}.ts ✅ -└── tests/… ✅ unit, integration, e2e (sandboxed HOME) -``` - -## User Journey - -```mermaid -flowchart TD - A[aidd telemetry ingest, or any telemetry command] --> B[Lock the ledger] - B --> C[Read new bytes of every transcript] - C --> D[Resolve each line's cwd to a repository] - D --> E{Repository opted in?} - E -- no --> F[Not stored] - E -- yes --> G[Upsert by key, fold, snapshot branch bindings, unlock] - D -- cwd gone, never seen alive --> H[Not stored, counted in coverage] -``` - -## Test Scope - -```mermaid ---- -title: Test scope ---- -journey - section Setup - Sandbox HOME with fixture transcripts and two git repos, one opted in => ready: 5: system - section Happy path - Run aidd telemetry ingest twice => the ledger holds each key once, second run adds nothing: 5: cli - section Edge case - concurrent ingests - Two ingests start together => both finish => no duplicate, no torn line: 1: cli - section Edge case - stale lock - A lock left by a dead pid => ingest => lock cleared and ingest completes: 1: cli - section Edge case - deleted working dir - cwd seen alive at an earlier ingest, now deleted => ingest => resolved from the persisted resolution, stored: 1: cli - cwd never seen alive => ingest => not stored, counted as unresolved in coverage: 1: cli - section Edge case - refusal - AIDD_TELEMETRY=0, or the project not opted in => ingest => nothing stored for it: 1: cli - section Teardown - Remove the sandbox => baseline restored: 5: system -``` - -## Tasks to do - -### `1)` Repository resolution - -> Each line is tied to a repository at ingest, from its own cwd. - -1. realpath + case-fold the cwd, `git rev-parse --show-toplevel --git-common-dir`, cached per cwd. -2. Repository id: hash of `host/owner/repo` parsed from `origin` (https and ssh, credentials stripped), else the root commit SHA; the raw url is never stored. -3. Every resolution `cwd -> {repository id, root, consent}` is persisted in `/ledger/roots.json` the first time the cwd is seen alive, so lines from a worktree deleted later still resolve. A cwd never seen alive is not stored (its consent is unprovable) and is counted in coverage. Outside any repo: not stored, counted as `outside-repo` in coverage. -4. Snapshot every opted-in repository's branch bindings (git config keys plus the branch creation time read from its reflog at that moment) into `/bindings/branches.jsonl`, append-only, on each ingest and each declaration. Attribution reads only snapshots, so a deleted or renamed branch keeps its past attribution. - -### `2)` The ledger - -> Append-only, idempotent, safe under concurrency. - -1. `/ledger/.jsonl` by record `at`; offsets in `/ledger/offsets.json`. -2. Upsert by `(tool, key)` with the phase 3 fold; a newline guard before each append. -3. Lock file with pid and creation time; a lock whose pid is gone or older than its timeout is cleared. -4. Directory 0700 and files 0600 on POSIX, icacls on Windows, as `auth-storage` does. - -### `3)` Consent and ingest - -> Only opted-in projects are read into the ledger. - -1. Consent per repository root: `.aidd/config.json` `telemetry.enabled === true` **and** `telemetry.version === 2`. The previous version's bare `enabled: true` is not consent. `AIDD_TELEMETRY=0` refuses everything. Consent is checked when the cwd is first resolved. -2. `ingest` is the one use case; every later telemetry command calls it first. - -## Test acceptance criteria - -| Task | Acceptance criteria | -| ---- | ------------------- | -| 1 | https, ssh and credential-bearing remotes of one repo give one id; a repo without origin gets its root commit; a linked worktree resolves to its main repository's id | -| 2 | Re-ingesting the same transcripts never changes the ledger; two concurrent ingests leave each key once | -| 2 | A transcript rewritten shorter is re-read whole and the fold keeps the best record | -| 3 | A project without consent, and any project under `AIDD_TELEMETRY=0`, stores nothing | -| 1 | A branch deleted after its declaration still attributes its past records, read from the snapshot | -| all | Each rule (key fold, lock clearing, consent refusal, snapshot) has a named mutation that turns its test red | diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-5.md b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-5.md deleted file mode 100644 index da6df67d9..000000000 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-5.md +++ /dev/null @@ -1,89 +0,0 @@ ---- -status: implemented ---- - - - -# Instruction: Declare, the task command and its stores - -## Architecture projection - -> Tree of the final files. ✅ create · ✏️ modify · ❌ delete - -```txt -cli/src/ -├── presentation/commands/telemetry.ts ✏️ `task [--ticket ] | --none`, `task` alone shows the current binding -└── contexts/telemetry/ - ├── domain/task-declaration.ts ✅ name, ticket, none, declared_at, by (command | hook-intercept) - ├── domain/ports/{branch-binding-store,session-binding-store}.ts ✅ - ├── application/declare-task-use-case.ts ✅ - └── infrastructure/ - ├── branch-binding-store-adapter.ts ✅ git config branch..aiddTask / aiddTicket / aiddDeclaredAt - └── session-binding-store-adapter.ts ✅ /bindings/sessions.jsonl, append-only -scripts/__tests__/fixtures/telemetry-bindings/ ✅ the shared format fixture (read by CLI and hook tests) -``` - -## User Journey - -```mermaid -flowchart TD - A[Person runs aidd telemetry task checkout-fix --ticket PROJ-12] --> B{Inside a Claude session?} - B -- yes --> C[Bind that session from now] - B -- no --> D[No session to bind] - C --> E{On a working branch?} - D --> E - E -- yes, not yet bound --> F[Bind the branch] - E -- default branch or detached --> G[Branch untouched] - F --> H[Print what was bound] - G --> H -``` - -## Test Scope - -```mermaid ---- -title: Test scope ---- -journey - section Setup - Sandbox repo on feat/x, CLAUDE_CODE_SESSION_ID set => ready: 5: system - section Happy path - aidd telemetry task checkout-fix --ticket PROJ-12 => session and branch bound, both printed: 5: cli - aidd telemetry task => shows the binding and where it comes from: 5: cli - section Edge case - none - aidd telemetry task --none => declared none, never asked again on this branch: 1: cli - section Edge case - redeclare - A second declaration in the same session => applies from its own time, the first keeps the earlier work: 1: cli - section Edge case - default branch - On the default branch => only the session is bound, git config untouched: 1: cli - section Edge case - outside a session - No session id in env => branch bound only: 1: cli -``` - -## Tasks to do - -### `1)` The declaration and its two stores - -> One declaration, stored where its scope lives. - -1. Ticket is free text, trimmed, never parsed or validated against a forge. -2. Branch store: `git config --local branch..aiddTask|aiddTicket|aiddDeclaredAt`, then an immediate snapshot (phase 4, task 1.4) with the branch creation time read from the reflog now, before it can expire. -3. Session store: one JSON line per declaration `{session_id, task, ticket, none, declared_at, by}`; the latest per session wins from its own time. -4. Default branch (`origin/HEAD`, else `main`/`master` when no remote head) and detached HEAD never get a branch binding. - -### `2)` The shared format fixture - -> One file both languages test against. - -1. Fixture holds sample `sessions.jsonl`, `carries.jsonl` (phase 6) and the git config keys, with the expected "is this session bound, to what" answers, plus cases for every rule the hook duplicates: telemetry-dir resolution, default-branch detection, consent, and the Claude-only guard. -2. A CLI unit test reads it; phase 6's hook test reads the same file. - -## Test acceptance criteria - -| Task | Acceptance criteria | -| ---- | ------------------- | -| 1 | After a declaration on `feat/x`, `git config branch.feat/x.aiddTask` returns the task; renaming the branch keeps it | -| 1 | On the default branch or detached HEAD, git config is unchanged | -| 1 | Two declarations in one session are both kept, each with its own time | -| 2 | Changing a field name in the CLI writer without updating the fixture turns the fixture test red | -| all | Default-branch exclusion, latest-wins per session, and the none declaration each have a named mutation that turns their test red | diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-6.md b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-6.md deleted file mode 100644 index 25ec448a8..000000000 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-6.md +++ /dev/null @@ -1,126 +0,0 @@ ---- -status: implemented ---- - - - -# Instruction: Ask, the plugin hooks - -## Architecture projection - -> Tree of the final files. ✅ create · ✏️ modify · ❌ delete - -```txt -plugins/aidd-telemetry/hooks/ -├── hooks.json ✅ SessionStart (sync: session fact, clear carry), SessionStart (async: ingest), UserPromptSubmit -├── session-start.cjs ✅ records {pid, session, source}; on clear, carries the predecessor's session binding -├── catch-up.cjs ✅ async: runs `aidd telemetry ingest` if aidd is on PATH, prints nothing -├── prompt-gate.cjs ✅ intercept a typed declaration, else ask when the rule says so -└── lib/ - ├── claude-only.cjs ✅ payload session_id === CLAUDE_CODE_SESSION_ID, Claude-shaped transcript path - ├── presence.cjs ✅ ATTENDED=1 and ENTRYPOINT cli or claude-desktop, else absent - ├── binding.cjs ✅ reads git config + sessions.jsonl + carries.jsonl, answers "bound to what" - └── telemetry-dir.cjs ✅ AIDD_TELEMETRY_DIR, else AIDD_USER_CONFIG_DIR/telemetry, XDG, ~/.config/aidd/telemetry -scripts/__tests__/aidd-telemetry-hooks.test.js ✅ spawns each hook with payloads and a sandbox env -``` - -## User Journey - -```mermaid -flowchart TD - P[Person submits a prompt] --> Q{Claude, opted in, person present?} - Q -- no --> R[Pass, silent] - Q -- yes --> S{Prompt is aidd telemetry task ...?} - S -- yes --> T[Run the CLI declaration, block with the result, original prompt suppressed] - S -- no --> U{Working branch and nothing bound?} - U -- no --> R - U -- yes --> V[Block: declare a task, two ways shown] - C[/clear] --> W[Session fact: same pid, new session] - W --> X[Copy predecessor's session binding as carried, announce it] -``` - -## Test Scope - -```mermaid ---- -title: Test scope ---- -journey - section Setup - Temp repo on feat/x opted in, sandbox telemetry dir, Claude-shaped payloads => ready: 5: system - section Happy path - Prompt on unbound feat/x with ATTENDED=1 ENTRYPOINT=cli => blocked, reason names the command: 5: system - Prompt "aidd telemetry task fix-cart --ticket PROJ-1" => CLI called, blocked with confirmation, original prompt suppressed: 5: system - Next prompt => passes: 5: system - section Edge case - unattended - ATTENDED=0 or ENTRYPOINT=sdk-cli or variable absent => passes: 1: system - section Edge case - not Claude - Codex-shaped payload, or session_id differing from CLAUDE_CODE_SESSION_ID => passes, writes nothing: 1: system - section Edge case - default branch and detached - On main or detached HEAD => passes: 1: system - section Edge case - clear - SessionStart clear with the same CLAUDE_PID => carry written, systemMessage names the task: 1: system - section Edge case - opted out - Project not opted in, or AIDD_TELEMETRY=0 => every hook passes and writes nothing: 1: system - section Edge case - aidd missing - Declaration typed with no aidd on PATH => blocked with a message saying aidd is needed: 1: system -``` - -## Tasks to do - -### `1)` Guards shared by every hook - -> Inert outside Claude, outside opted-in projects, and when presence is unknown. - -1. Order for latency: env checks first (Claude-only, presence), then one file read (consent), then git. A person with the plugin but no consent pays no git call. -2. Claude-only: payload `session_id` equals `CLAUDE_CODE_SESSION_ID` and the transcript path is under a Claude `projects/` directory (separators normalised). -3. Consent read from `.aidd/config.json` at the repo root, `AIDD_TELEMETRY=0` refuses; the same rule as the CLI, pinned by a shared fixture case. -4. Every hook exits 0 on any internal error, except the deliberate block; a crash never blocks a prompt. - -### `2)` The prompt gate - -> Ask once per working branch, answer without the model. - -1. A prompt matching `^\s*!?\s*aidd telemetry task\b` is tokenized by a strict parser (no shell syntax accepted), and the CLI is spawned with an argv array and no shell, `aidd.cmd` resolved explicitly on Windows; it then blocks with the CLI's output and `suppressOriginalPrompt: true`. -2. Otherwise block only when: Claude, opted in, present, working branch, neither the session (declared or carried) nor the branch is bound, and the answer path works: `aidd` resolves and `aidd telemetry task --help` succeeds (checked only on this rare path). Otherwise pass silently: a block nobody can answer is never raised. -3. The reason names both answers: `! aidd telemetry task [--ticket ]` in a terminal, the same text as a prompt anywhere. - -### `3)` Session facts and the clear carry - -> `/clear` keeps the task, visibly. - -1. On every SessionStart, append `{pid, session_id, source, at}` to the pid's history, pid from `CLAUDE_PID` (undocumented: pinned by a test, falling back to the parent pid as measured, and to no carry when neither is available). -2. On `source: clear`, take the pid's latest session before this one; if it is bound (declared or carried), append `{session_id, from, at}` to `carries.jsonl` and print a `systemMessage` naming the task and how to change it. -3. `resume` uses the resumed session's own binding; `fork` carries from the session it forked from when the history shows it (same work), probed once on `/branch`; `startup` and `compact` carry nothing. - -### `4)` Async catch-up - -> Ingest at session start, never visible, never blocking. - -1. A separate `async: true` SessionStart entry runs `aidd telemetry ingest` when `aidd` resolves on PATH; prints nothing to stdout. -2. Assert that `async: true` survives both Claude install routes (marketplace copy and the flat settings merge). - -### `5)` Presence on non-terminal surfaces - -> Know, not assume, what the IDE and desktop app expose. - -1. Probe once, with the maintainer, in the VS Code extension and the desktop app: `CLAUDE_CODE_SESSION_ATTENDED`, `CLAUDE_CODE_ENTRYPOINT`, and whether a block reason displays. Record the values in `context/target.md`. -2. Extend the presence rule only with values observed there; until observed, those surfaces never block. - -## Test acceptance criteria - -| Task | Acceptance criteria | -| ---- | ------------------- | -| 1 | A Codex-shaped payload and a mismatched session id both pass and write no file | -| 1 | A hook given malformed JSON exits 0 and passes the prompt | -| 2 | An unbound working branch with a person present blocks; the same with ATTENDED=0 passes | -| 2 | A typed declaration reaches no model: the hook blocks and the CLI has stored it | -| 3 | After a clear on a bound session, the new session reads as bound to the same task, marked carried | -| 3 | The hook reads the phase 5 shared fixture and returns the same answers as the CLI test | -| 4 | The catch-up writes nothing to stdout and exits 0 when aidd is absent | -| 4 | A flat-merged `.claude/settings.json` keeps `async: true` on the catch-up entry | -| 2 | With `aidd` absent or lacking `telemetry task`, an unbound branch is never blocked | -| 2 | A typed declaration containing `;`, `&&`, a pipe or backticks runs no second command | -| 5 | Observed IDE and desktop values are recorded, with a test per observed value | -| all | Claude-only guard, presence rule, answer-path check and clear carry each have a named mutation that turns their test red | -| all | The suite passes on the Windows CI job | diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-7.md b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-7.md deleted file mode 100644 index abfee804e..000000000 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-7.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -status: implemented ---- - - - -# Instruction: Attribute and report - -## Architecture projection - -> Tree of the final files. ✅ create · ✏️ modify · ❌ delete - -```txt -cli/src/ -├── presentation/commands/telemetry.ts ✏️ `report [--from --to | --days] [--axis ] [--json]` -├── presentation/display/telemetry-report-display.ts ✅ text rendering only -└── contexts/telemetry/ - ├── domain/attribution.ts ✅ pure: record + bindings + carries + branch facts -> task/ticket or unattributed(reason) - ├── domain/usage-report.ts ✅ totals and axes, four counters apart - ├── domain/person.ts ✅ opted-in person id or none - ├── domain/ports/person-identity.ts ✅ - ├── application/report-usage-use-case.ts ✅ ingest, then attribute and aggregate - └── infrastructure/person-identity-adapter.ts ✅ userConfigDir()/telemetry/identity.json -``` - -## User Journey - -```mermaid -flowchart TD - A[aidd telemetry report --days 7] --> B[Ingest] - B --> C[Attribute every record] - C --> D[Aggregate by axis] - D --> E[Totals, one row per value, unattributed rows with reasons, coverage line] -``` - -## Test Scope - -```mermaid ---- -title: Test scope ---- -journey - section Setup - Ledger fixture spanning two days, two models, two branches, one carry, one redeclaration => ready: 5: system - section Happy path - report --axis task => one row per task, unattributed rows by reason, sum equals total: 5: cli - report on every axis => each sums to the same total, four counters apart: 5: cli - section Edge case - retroactive branch - Records on feat/x before it is declared => report => unattributed no-binding; declare; report => on the task: 1: cli - section Edge case - carried then corrected - A carried session later declared => report => the whole session on the declared task: 1: cli - section Edge case - mid-session redeclaration - Second declaration at t => report => before t on the first task, after t on the second: 1: cli - section Edge case - unknown counters - A record with an unknown counter => report => that counter shown unknown for the row, never summed as zero: 1: cli -``` - -## Tasks to do - -### `1)` Attribution - -> One pure function, the command the only source. - -1. Order: session declaration effective at the record's time; else the session's carry (whole session until its first declaration); else the branch binding read from the snapshots, when the record's `git_branch` and repository match and its time is after the branch creation time captured in the snapshot; else unattributed. -2. A first declaration in a carried session applies to the whole session; later ones apply from their own time. -3. Unattributed reasons: `outside-repo`, `root-unresolved`, `no-binding`, `declared-none`. -4. Subagent and advisor records inherit their parent session's attribution through `session_id`. - -### `2)` The report - -> Every axis reconciles to one total. - -1. Axes: `total`, `person`, `session`, `model`, `day` (UTC), `repository`, `task`, `ticket`. -2. Four counters per row, and a row total; an unknown counter is shown as unknown for its row and excluded from no total silently: the report states how many records had an unknown. -3. A coverage line: files read, records, unrecognised shapes, and the oldest transcript date still on disk. -4. `--json` emits a versioned envelope documented in phase 9. - -### `3)` Person - -> Opt-in, fresh, local. - -1. `aidd telemetry identity ` writes `{person_id}` under the telemetry dir; `--off` removes it; absent means no person axis value. - -## Test acceptance criteria - -| Task | Acceptance criteria | -| ---- | ------------------- | -| 1 | Each attribution rule has a unit test and a named mutation that turns it red | -| 1 | Declaring a branch after its work moves that work from `no-binding` to the task on the next report | -| 2 | For every axis, the rows sum to the total on the fixture and on a property-based generated ledger | -| 2 | The JSON envelope carries its version, and the text never shows a path, cwd or branch name it does not need | -| 3 | Without an identity file, the person axis shows one "not set" row; with one, the id | diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-8.md b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-8.md deleted file mode 100644 index 5ec4b0e38..000000000 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-8.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -status: implemented ---- - - - -# Instruction: Opt in, opt out, forget, and what V1 left behind - -## Architecture projection - -> Tree of the final files. ✅ create · ✏️ modify · ❌ delete - -```txt -cli/src/ -├── presentation/commands/telemetry.ts ✏️ `on [--yes]`, `off`, `forget [--yes]` -└── contexts/telemetry/ - ├── domain/legacy-leftovers.ts ✅ what V1 wrote, by location - ├── application/{telemetry-on,telemetry-off,forget-telemetry}-use-case.ts ✅ - └── infrastructure/legacy-leftovers-adapter.ts ✅ finds and removes them -``` - -## User Journey - -```mermaid -flowchart TD - A[aidd telemetry on] --> B[Write telemetry.enabled] - B --> C[Remove V1 leftovers in this repo] - C --> D[Report Claude transcript retention and the one-line fix] - E[aidd telemetry forget] --> F[Show what would go] - F -- --yes --> G[Remove ledger, bindings, identity, V1 sink and identity] -``` - -## Test Scope - -```mermaid ---- -title: Test scope ---- -journey - section Setup - Sandbox repo carrying V1 leftovers (trailer line + delegate, aidd_docs/runs, ignore entry) and a V1 sink + identity => ready: 5: system - section Happy path - aidd telemetry on --yes => enabled, trailer line and delegate gone together, runs dir and ignore entry gone: 5: cli - git commit => succeeds, no AIDD-Session-Id trailer: 5: cli - aidd telemetry forget --yes => ledger, bindings, identity, V1 day files and V1 identity removed: 5: cli - section Edge case - foreign hook content - prepare-commit-msg has other lines => on => only the trailer line removed, the rest intact: 1: cli - section Edge case - hook managers - lefthook or husky manages the hook => on => the delegate is kept if a manager line still calls it, and the manual step is printed: 1: cli - section Edge case - forget preview - forget without --yes => lists what would go, removes nothing: 1: cli - section Edge case - retention - Claude cleanupPeriodDays unset or below 3650 => on => says so and prints the setting to add, writes nothing in the Claude profile: 1: cli - section Teardown - Remove the sandbox => baseline restored: 5: system -``` - -## Tasks to do - -### `1)` Opt in and V1 leftovers - -> Enabling leaves the repository as if V1 had never run. - -1. `on` writes `.aidd/config.json` `telemetry: {enabled: true, version: 2}`, keeping other keys; `--yes` skips the confirmation. A repository carrying only the previous version's `enabled: true` stays off until `on` runs. -2. Remove the `prepare-commit-msg` line calling `aidd-session-trailer.sh` and the delegate script in the same step; never remove the script while a line still calls it. -3. Remove `aidd_docs/runs/` (V1 journal, git-ignored) and its `.gitignore` entry. -4. Read Claude's effective `cleanupPeriodDays` (user and project settings, read only) and print the fix when it is below 3650. - -### `2)` Opt out and forget - -> Off stops reading; forget erases. - -1. `off` sets `telemetry.enabled: false`, keeping `version`; nothing else changes. -2. `forget` previews by default; `--yes` removes the ledger, bindings, carries, identity, V1 day files at the telemetry dir root and V1's `identity.json`, and the git config `aiddTask|aiddTicket|aiddDeclaredAt` keys of every still-existing repository the binding snapshots name. -3. The previous version's locations come from phase 1's notes: sink dir under `AIDD_TELEMETRY_DIR`, `AIDD_USER_CONFIG_DIR/telemetry`, `%APPDATA%\aidd\telemetry` on Windows, `~/.config/aidd/telemetry`; identity at `%APPDATA%\aidd\identity.json` or `~/.config/aidd/identity.json`. - -## Test acceptance criteria - -| Task | Acceptance criteria | -| ---- | ------------------- | -| 1 | After `on` in a V1 repo, a commit succeeds and carries no AIDD-Session-Id trailer | -| 1 | A prepare-commit-msg with other content keeps that content byte for byte | -| 1 | Nothing under the real or sandboxed Claude profile is written | -| 2 | `forget` without `--yes` changes nothing on disk; with it, nothing of either version remains in the telemetry dir | -| all | Trailer line-and-script pairing, foreign-content preservation and forget's preview each have a named mutation that turns their test red | diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-9.md b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-9.md deleted file mode 100644 index f8a9c7944..000000000 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/phase-9.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -status: implemented ---- - - - -# Instruction: Skills, docs, and the end-to-end proof - -## Architecture projection - -> Tree of the final files. ✅ create · ✏️ modify · ❌ delete - -```txt -. -├── plugins/aidd-telemetry/ -│ ├── skills/00-init/ ✅ opt in, identity, forget, through the CLI -│ ├── skills/01-usage/ ✅ answer "how many tokens" questions with `aidd telemetry report` -│ ├── README.md ✏️ what it measures, what it asks and when, what stays local -│ └── CATALOG.md ✏️ regenerated -├── aidd_docs/product/usage-contract.md ✅ the usage record, the report envelope, the binding stores -├── cli/aidd_docs/memory/telemetry.md ✅ how the context is built -├── memory, README, docs/*.md ✏️ the new telemetry, one line each where relevant -└── cli/tests/e2e/telemetry-journey.e2e.test.ts ✅ the full journey on the built binary -``` - -## User Journey - -```mermaid -flowchart TD - A[Opt in] --> B[Work: asked once on a working branch] - B --> C[/clear, keep working, carried/] - C --> D[Headless run on the same branch] - D --> E[Report: per task, ticket, model, day, session, person, repository] - E --> F[Every axis equals the total] -``` - -## Test Scope - -```mermaid ---- -title: Test scope ---- -journey - section Setup - Built binary, sandbox HOME and CLAUDE_CONFIG_DIR, a repo on feat/x, synthetic transcripts for three sessions => ready: 5: cli - section Happy path - on, hook block, typed declaration, clear carry, headless session, report --json => each axis sums to the total, tokens land on the declared task and ticket: 5: cli - section Edge case - real history oracle - Ingest a copy of local history in a sandbox => daily totals equal an independent count of the same transcripts, the reference's cache-write under-count explained: 1: cli - section Teardown - Delete the sandbox => no file outside it touched: 5: system -``` - -## Tasks to do - -### `1)` Skills - -> Two skills, both thin over the CLI, saying loudly when it is absent. - -1. `00-init`: opt in after consent, set or remove the identity, forget with a preview. -2. `01-usage`: map a question to `aidd telemetry report` flags and read the JSON; never compute a number itself. - -### `2)` Docs - -> One contract, written once. - -1. `aidd_docs/product/usage-contract.md`: the record, the envelope version, the stores and their owners, the ask rule, the carry rule. -2. Plugin README, CLI memory and repo memory lines. -3. Set the bundle budget once, to the measured build plus about 2 %, with its registry line. - -### `3)` Proof - -> The whole journey on the built binary, and one comparison on real data. - -1. One e2e journey covering the spec's done-when list, no AI tool binary on PATH. -2. A manual, documented oracle run on a copied history in a sandbox, comparing daily totals with an independent count; its output kept in this folder, with no personal data. - -## Test acceptance criteria - -| Task | Acceptance criteria | -| ---- | ------------------- | -| 1 | Each skill's actions name only commands that exist in the CLI help golden | -| 2 | Every doc link resolves; no doc mentions the previous telemetry's journal, envelope or commands | -| 3 | The e2e journey passes on Linux, macOS and Windows CI | -| 3 | The oracle note shows equal totals for every compared day, or each difference explained to the token | diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/plan.md b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/plan.md index 1905318ee..a56b78d01 100644 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/plan.md +++ b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/plan.md @@ -12,25 +12,27 @@ status: implemented | Field | Value | | ---------- | --------------------------------------------------------- | | **Goal** | Replace the previous telemetry with a deterministic, declaration-attributed token count for Claude Code | -| **Source** | [`spec.md`](./spec.md), with [`context/target.md`](./context/target.md) and [`context/usage-record-contract.md`](./context/usage-record-contract.md) | +| **Source** | [`spec.md`](./spec.md), with [`context/target.md`](./context/target.md) | ## Phases -| # | Phase | File | -| --- | -------------------------------------------------- | ---------------------------- | -| 1 | Remove the previous telemetry from the CLI | [`phase-1.md`](./phase-1.md) | -| 2 | Remove it from the plugins, scripts, CI and docs | [`phase-2.md`](./phase-2.md) | -| 3 | Count: Claude transcripts to usage records | [`phase-3.md`](./phase-3.md) | -| 4 | Store: the ledger, ingest, consent | [`phase-4.md`](./phase-4.md) | -| 5 | Declare: the task command and its stores | [`phase-5.md`](./phase-5.md) | -| 6 | Ask: the plugin hooks | [`phase-6.md`](./phase-6.md) | -| 7 | Attribute and report | [`phase-7.md`](./phase-7.md) | -| 8 | Opt in, opt out, forget, and what V1 left behind | [`phase-8.md`](./phase-8.md) | -| 9 | Skills, docs, and the end-to-end proof | [`phase-9.md`](./phase-9.md) | +| # | Phase | +| --- | -------------------------------------------------- | +| 1 | Remove the previous telemetry from the CLI | +| 2 | Remove it from the plugins, scripts, CI and docs | +| 3 | Count: Claude transcripts to usage records | +| 4 | Store: the ledger, ingest, consent | +| 5 | Declare: the task command and its stores | +| 6 | Ask: the plugin hooks | +| 7 | Attribute and report | +| 8 | Opt in, opt out, forget, and what V1 left behind | +| 9 | Skills, docs, and the end-to-end proof | Phases 1 and 2 land as their own commits but never ship alone: the branch merges only once phase 9 is green, so `next` never carries a CLI without measurement. +The per-phase files and the per-commit mutation log were removed once the work merged; git history keeps them. + ## Resources | Source | Verified | diff --git a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/spec.md b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/spec.md index 3f89444a6..f413cb1d8 100644 --- a/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/spec.md +++ b/aidd_docs/tasks/2026_10/2026_10_09_deterministic-tokens/spec.md @@ -1,5 +1,5 @@ --- -status: framed +status: done branch: feat/telemetry-deterministic-tokens --- @@ -105,7 +105,6 @@ None. ## Context - Measurements, verified behaviours and the target shape: [`context/target.md`](context/target.md). -- The common usage record and per-tool normalisation: [`context/usage-record-contract.md`](context/usage-record-contract.md). -- Prototype readers and their mutation-proved fixtures: [`context/prototype/`](context/prototype/). +- The common usage record, as shipped: [usage contract](../../../product/usage-contract.md). The draft record and the Python prototype readers were removed after the merge; git history keeps them. - Decision records kept: [hosted destination](../../../memory/internal/decisions/measurement-may-reach-a-hosted-destination.md), [price table](../../../memory/internal/decisions/a-price-table-is-the-destination-s.md). diff --git a/aidd_docs/tasks/2026_10/2026_10_15_ledger-by-day/contract.md b/aidd_docs/tasks/2026_10/2026_10_15_ledger-by-day/contract.md index 0f5413c82..84dd7e1c8 100644 --- a/aidd_docs/tasks/2026_10/2026_10_15_ledger-by-day/contract.md +++ b/aidd_docs/tasks/2026_10/2026_10_15_ledger-by-day/contract.md @@ -1,5 +1,5 @@ --- -status: in-progress +status: done issue: 1001 --- diff --git a/aidd_docs/tasks/2026_10/2026_10_17_transcripts-are-read-as-a-stream/evidence.md b/aidd_docs/tasks/2026_10/2026_10_17_transcripts-are-read-as-a-stream/evidence.md index e32aaac18..e97831319 100644 --- a/aidd_docs/tasks/2026_10/2026_10_17_transcripts-are-read-as-a-stream/evidence.md +++ b/aidd_docs/tasks/2026_10/2026_10_17_transcripts-are-read-as-a-stream/evidence.md @@ -51,7 +51,7 @@ Generated transcript of synthetic lines (fake ids, about 2 KB each, one usage li | this branch | 158,520,954 B (151 MB) | 1,580 | 158,520,954 | 69,402,624 B (66 MB) | | this branch | 421,389,954 B (402 MB) | 4,200 | 421,389,954 | 80,478,208 B (77 MB) | -Rerun with [`memory-bench.mjs`](./memory-bench.mjs) after review round 1 (`node memory-bench.mjs []`, peak RSS of the reading process from `process.resourceUsage().maxRSS`; `origin/next` measured from its source extracted with `git archive`), `TMPDIR` in the scratchpad, files deleted by the script: +Rerun with `memory-bench.mjs` after review round 1 (the bench was removed after the merge, the figures below are what it measured; peak RSS of the reading process from `process.resourceUsage().maxRSS`; `origin/next` measured from its source extracted with `git archive`), `TMPDIR` in the scratchpad, files deleted by the script: | Run | Transcript | Records | Offset saved | Peak RSS | | --- | --- | --- | --- | --- | @@ -101,7 +101,7 @@ Four minors from [`review.md`](./review.md), applied. | 1 | The memory bound was overstated | usage contract and `telemetry.md`: a block, up to about three copies of the longest line, every record and every unrecognised line naming a directory | | 2 | Every line copied by `Buffer.concat` | a line whole in one block is decoded in place | | 3 | A file rewritten in place between two blocks could splice a line and save a wrong offset | after the last block, `fstat` of the handle: shorter than when the read began, and the read returns no position (the path check added here was dropped in round 2, see below); `ReadUsageUseCase` keeps none of that file's records, lines or position, so ingest keeps the one it had. A rewrite to the same size or larger is not seen: residual limit in the usage contract | -| 4 | The bench was not kept | [`memory-bench.mjs`](./memory-bench.mjs), rerun above | +| 4 | The bench was not kept | `memory-bench.mjs`, rerun above, then removed after the merge: probes stay out of task folders | ### Red run diff --git a/aidd_docs/tasks/2026_10/2026_10_17_transcripts-are-read-as-a-stream/memory-bench.mjs b/aidd_docs/tasks/2026_10/2026_10_17_transcripts-are-read-as-a-stream/memory-bench.mjs deleted file mode 100644 index 3755ad68c..000000000 --- a/aidd_docs/tasks/2026_10/2026_10_17_transcripts-are-read-as-a-stream/memory-bench.mjs +++ /dev/null @@ -1,108 +0,0 @@ -#!/usr/bin/env node -// Peak memory of reading one generated transcript through ReadUsageUseCase and the real adapter. -// -// node memory-bench.mjs [] -// -// The source defaults to this repository's `cli/src`. To measure another revision, extract its -// source first, e.g. `git archive origin/next cli/src | tar -x -C /tmp/old`, then pass -// `/tmp/old/cli/src`. The transcript is synthetic (fake ids, ~2 KB lines that are no usage line, -// one usage line in 50), written to a temporary directory and deleted afterwards. The read runs -// in its own `node` process, whose peak RSS is printed. Needs `pnpm install` in `cli/`. -import { spawnSync } from "node:child_process"; -import { closeSync, mkdirSync, mkdtempSync, openSync, rmSync, writeFileSync, writeSync } from "node:fs"; -import { createRequire } from "node:module"; -import { tmpdir } from "node:os"; -import { dirname, join, resolve } from "node:path"; -import { fileURLToPath } from "node:url"; - -const here = dirname(fileURLToPath(import.meta.url)); -const cli = resolve(here, "../../../../cli"); -const megabytes = Number(process.argv[2]); -const src = resolve(process.argv[3] ?? join(cli, "src")); -if (!(megabytes > 0)) { - console.error("usage: node memory-bench.mjs []"); - process.exit(1); -} - -const work = mkdtempSync(join(tmpdir(), "aidd-memory-bench-")); -try { - const entry = join(work, "entry.ts"); - writeFileSync( - entry, - `import { ReadUsageUseCase } from ${JSON.stringify(`${src}/contexts/telemetry/application/read-usage-use-case.js`)}; -import { readClaudeUsageLine } from ${JSON.stringify(`${src}/contexts/telemetry/domain/formats/claude-transcript-usage.js`)}; -import { ClaudeTranscriptSourceAdapter } from ${JSON.stringify(`${src}/contexts/telemetry/infrastructure/claude-transcript-source-adapter.js`)}; -const reading = await new ReadUsageUseCase( - new ClaudeTranscriptSourceAdapter(process.argv[2]), - readClaudeUsageLine -).execute(new Map()); -const [position] = [...reading.positions.values()]; -console.log(JSON.stringify({ - records: reading.records.length, - offset: position?.offset, - maxRssMB: Math.round(process.resourceUsage().maxRSS / 1024), -})); -` - ); - // esbuild is the CLI's bundler, reached through tsup, which depends on it. - const fromCli = createRequire(join(cli, "package.json")); - const esbuild = createRequire(fromCli.resolve("tsup/package.json"))("esbuild"); - const bundle = join(work, "bench.mjs"); - esbuild.buildSync({ - entryPoints: [entry], - bundle: true, - platform: "node", - format: "esm", - outfile: bundle, - logLevel: "warning", - }); - - const projects = join(work, "projects"); - mkdirSync(join(projects, "p"), { recursive: true }); - const fd = openSync(join(projects, "p", "s.jsonl"), "w"); - const pad = "lorem ipsum ".repeat(160); - let written = 0; - for (let n = 0; written < megabytes * 2 ** 20; ) { - const lines = []; - for (let i = 0; i < 1000; i += 1, n += 1) { - lines.push( - JSON.stringify( - n % 50 === 0 - ? { - type: "assistant", - sessionId: "s-bench", - requestId: `req_${n}`, - timestamp: "2026-10-08T12:00:00.000Z", - cwd: "/work/bench", - message: { - id: `msg_${n}`, - model: "m", - usage: { - input_tokens: 1, - output_tokens: 2, - cache_read_input_tokens: 3, - cache_creation_input_tokens: 4, - }, - }, - } - : { - type: "user", - sessionId: "s-bench", - timestamp: "2026-10-08T12:00:00.000Z", - cwd: "/work/bench", - message: { content: pad }, - } - ) - ); - } - written += writeSync(fd, `${lines.join("\n")}\n`); - } - closeSync(fd); - - const run = spawnSync(process.execPath, [bundle, projects], { encoding: "utf8" }); - if (run.status !== 0) throw new Error(run.stderr); - const result = JSON.parse(run.stdout); - console.log(JSON.stringify({ transcriptBytes: written, ...result })); -} finally { - rmSync(work, { recursive: true, force: true }); -} diff --git a/cli/aidd_docs/memory/cli.md b/cli/aidd_docs/memory/cli.md index 558ea2e06..a45792cf9 100644 --- a/cli/aidd_docs/memory/cli.md +++ b/cli/aidd_docs/memory/cli.md @@ -11,7 +11,7 @@ Thirty leaf commands. Read them live: `aidd --help`, then each group's. `scripts - `plugin install | list | remove | search | update`. - `marketplace add | list | remove | refresh | check`. - `auth login | logout | status`. -- `telemetry on | off | forget | ingest | task | report | identity`: local token measurement for Claude Code, mapped in `telemetry.md`. +- `telemetry on | off | forget | ingest | task | status | report | identity`: local token measurement for Claude Code, mapped in `telemetry.md`. - `translate `: author-side, converts a source into a target-native plugin tree. - `update`, aliased `upgrade`: the CLI itself. diff --git a/cli/aidd_docs/memory/telemetry.md b/cli/aidd_docs/memory/telemetry.md index 616f9382c..d5e8a57eb 100644 --- a/cli/aidd_docs/memory/telemetry.md +++ b/cli/aidd_docs/memory/telemetry.md @@ -11,7 +11,7 @@ | one reader in the ingest list | `runtime/wiring/telemetry.ts` | | its name | `USAGE_TOOLS` in `domain/usage-record.ts` | -Storage, attribution and the report need nothing. Session declarations come from Claude's hooks only (release 1): another tool's calls are attributed by branch. +Storage, attribution and the report need nothing. Task and ticket come only from `aidd telemetry task`, whoever runs it; the plugin's hooks only invoke it from Claude Code, so another tool's calls are attributed by branch. Rules: [usage contract](../../../aidd_docs/product/usage-contract.md#attribution). ## Data flow @@ -30,7 +30,7 @@ flowchart LR ``` - A command that reports or ingests reads new lines first, under the ledger lock. `ingest` is the same step on its own, run quietly by the plugin at session start. -- A transcript is read a block at a time (1 MiB), each complete line handed on as it is found, never the unread tail whole. An ingest holds a block, up to about three copies of the longest line at once (its pieces, the joined bytes, the decoded string), and what the reading keeps: every record and every unrecognised line naming a directory. An unterminated last line waits for the next read. A transcript that shrank while it was read counts none of that read and keeps its position, to be read again. One replaced at its path meanwhile is read as it was opened, whole. +- A transcript is read a block at a time (1 MiB), each complete line handed on as it is found, never the unread tail whole. An ingest holds a block, up to about three copies of the longest line at once (its pieces, the joined bytes, the decoded string), and what the reading keeps: every record and every unrecognised line naming a directory. What a rewrite, a shrink or a replacement during a read does: [residual limits](../../../aidd_docs/product/usage-contract.md#residual-limits). - A report's envelope keeps its figures apart from its `run`: defined once in [the envelope reference](../../../plugins/aidd-telemetry/skills/01-usage/references/envelope.md#figures-and-run). - Reading is one pass: tool-specific only up to the usage record. Resolving, storing, attributing and reporting are written once for every tool. - The hooks and the CLI never call each other's code. They meet in the files, whose formats one fixture pins on both sides. @@ -54,7 +54,7 @@ flowchart LR | Where a call's repository comes from | `application/directory-resolver.ts`, `domain/repository-identity.ts` | | How data an earlier version stored is brought to today's key and id rules: read-time in every store reader, rewritten by the first ingest and again when a root left pending adds an id | `application/legacy/stored-usage-migration.ts`, `domain/legacy/` (key, id and table rules), `infrastructure/legacy/repository-alias-store-adapter.ts`; rules in the [usage contract](../../../aidd_docs/product/usage-contract.md#stores) | | Which task a call belongs to | `domain/report/attribution.ts`, `domain/declaration/binding-resolution.ts` | -| Whether a branch is declared: only by `task`, on that branch (`aiddDeclaredAt` set, `aiddDeclaredFor` absent or equal to the branch); the hook's twin is `plugins/aidd-telemetry/hooks/lib/git.cjs` | `domain/branch-binding.ts` `parseBranchConfig`; rule and limits in the [usage contract](../../../aidd_docs/product/usage-contract.md#branch-declaration) | +| Whether a branch is declared; the hook's twin is `plugins/aidd-telemetry/hooks/lib/git.cjs` | `domain/branch-binding.ts` `parseBranchConfig`; rule and limits in the [usage contract](../../../aidd_docs/product/usage-contract.md#branch-declaration) | | What counts as consent | `domain/telemetry-consent.ts`, `infrastructure/git-consent-adapter.ts` | | Whether a call is stored | `application/directory-resolver.ts` | | What a clone is | `domain/consent/clone-identity.ts` | @@ -66,7 +66,7 @@ flowchart LR | What `on` and `forget` clean up from an earlier measurement | `domain/legacy/`, `application/switch/`, `application/forget/` | | The report axes and the envelope | `domain/report/usage-report.ts`, `application/report/report-envelope.ts` | | Who the measurement names, and what `on` offers | `domain/identity/person-identity.ts`, `application/identity/`, `infrastructure/identity/git-user-source-adapter.ts`; the question is the `on` command's | -| What a report's `filters` select (`--task`, `--ticket`, `--person`, `--model`, `--agent`, `--session`, `--repository`, `--absent`, `--unattributed`), and where a start of an id or `.` is resolved | `domain/report/report-filter.ts`, `application/report/report-usage-use-case.ts` | +| What a report's `filters` select, and where a start of an id or `.` is resolved | `domain/report/report-filter.ts`, `application/report/report-usage-use-case.ts`; flags in the [usage contract](../../../aidd_docs/product/usage-contract.md#filters) | ## The plugin side diff --git a/cli/src/contexts/telemetry/application/ledger-upkeep.ts b/cli/src/contexts/telemetry/application/ledger-upkeep.ts index 1adbc29a4..557e31966 100644 --- a/cli/src/contexts/telemetry/application/ledger-upkeep.ts +++ b/cli/src/contexts/telemetry/application/ledger-upkeep.ts @@ -10,9 +10,8 @@ export interface Stored { } /** Folds incoming calls into the days around them, and rewrites only the days that changed, - * plus any partition read that was damaged or not in daily form, so a month is split and a - * line that is not a record put after the records, kept. Nothing outside the window is read, - * except a monthly file it splits and every day of that month, and every day when a call has no + * plus any day read that was damaged or unsettled, so a line that is not a record is put after + * the records, kept. Nothing outside the window is read, except every day when a call has no * request id. Runs under the caller's lock. */ export async function storeCalls( ledger: UsageLedger, @@ -28,9 +27,8 @@ export async function storeCalls( return { added: merged.added, updated: merged.updated, skippedLines: held.skippedLines }; } -/** Reads the partitions and rewrites, folded, each one read that is damaged or not in daily - * form: a monthly one, or one holding a call another read also holds. With no test, it reads - * every partition, and after it the ledger is byte-identical to a single ingest of the same +/** Reads the partitions and rewrites, folded, each one read that is damaged or unsettled (see + * `LoadedLedger`). With no test, it reads every partition, and after it the ledger is byte-identical to a single ingest of the same * calls. With a test on the day, it reads and settles only what the ledger loads for it (see * `UsageLedger.load`), and leaves every other partition as it is. A settled day is not * touched. Runs under the caller's lock. */ diff --git a/cli/src/contexts/telemetry/domain/ports/usage-ledger.ts b/cli/src/contexts/telemetry/domain/ports/usage-ledger.ts index 4eb7011c6..ab361b333 100644 --- a/cli/src/contexts/telemetry/domain/ports/usage-ledger.ts +++ b/cli/src/contexts/telemetry/domain/ports/usage-ledger.ts @@ -1,8 +1,7 @@ import type { StoredUsage } from "../stored-usage.js"; import type { TranscriptPosition } from "../transcript-position.js"; -/** A partition is named by its key: `YYYY-MM-DD` for a day, `YYYY-MM` for a month as an - * earlier version wrote it. */ +/** A partition is a day, named `YYYY-MM-DD`. */ export interface LoadedLedger { /** One record per `(tool, key)` across every partition read. */ readonly records: readonly StoredUsage[]; @@ -10,9 +9,9 @@ export interface LoadedLedger { readonly skippedLines: number; /** The partitions read that held one: saving them puts them in canonical order. */ readonly damaged: ReadonlySet; - /** The partitions read that are not in canonical daily form: every monthly one, and every - * one holding a call twice or a call another partition read also holds. Saving them settles - * them. */ + /** The partitions read that are not in canonical form: every one holding a call twice, a + * call another partition read also holds, or a call an earlier version keyed or named + * otherwise. Saving them settles them. */ readonly unsettled: ReadonlySet; } @@ -42,18 +41,14 @@ export interface UsageLedger { readonly leftOut: LeftOutStore; /** Runs `work` while no other process writes the ledger. */ exclusively(work: () => Promise): Promise; - /** Every partition, daily and monthly. Given `holds`, a test on a `YYYY-MM-DD` day, only the - * days it holds, plus every monthly partition it holds one of the calendar days of, read with - * every day of that month: a half-finished split may hold a better snapshot in a day. What a - * load costs is the partitions on disk, never how many days the test holds. */ + /** Every partition. Given `holds`, a test on a `YYYY-MM-DD` day, only the days it holds. What + * a load costs is the partitions on disk, never how many days the test holds. */ load(holds?: (day: string) => boolean): Promise; /** Makes the ledger hold exactly these records, every other partition removed. A partition * that already holds what it should is not touched. Given `partitions`, a day is written - * from `records`, or removed when they hold none for it; a month has each of its days - * written from `records`, each of its days on disk they hold none for removed, then its - * monthly partition removed, last. No other partition is read: the caller vouches that the - * rest is as it was. A line that is not a record is never removed: it stays in its day, - * after the records, in text order and once, and a month's goes to its first day. */ + * from `records`, or removed when they hold none for it. No other partition is read: the + * caller vouches that the rest is as it was. A line that is not a record is never removed: + * it stays in its day, after the records, in text order and once. */ save(records: readonly StoredUsage[], partitions?: ReadonlySet): Promise; positions(): Promise>; savePositions(positions: ReadonlyMap): Promise; diff --git a/cli/src/contexts/telemetry/infrastructure/usage-ledger-adapter.ts b/cli/src/contexts/telemetry/infrastructure/usage-ledger-adapter.ts index 9830de2b9..adf028848 100644 --- a/cli/src/contexts/telemetry/infrastructure/usage-ledger-adapter.ts +++ b/cli/src/contexts/telemetry/infrastructure/usage-ledger-adapter.ts @@ -5,12 +5,7 @@ import { readTextIfPresent } from "../../../kernel/reading/text-file.js"; import { storedNow } from "../domain/legacy/legacy-usage-key.js"; import type { PrivateStorage } from "../domain/ports/private-storage.js"; import type { LeftOutStore, LoadedLedger, UsageLedger } from "../domain/ports/usage-ledger.js"; -import { - parseStoredUsage, - partitionByDay, - type StoredUsage, - shiftDay, -} from "../domain/stored-usage.js"; +import { parseStoredUsage, partitionByDay, type StoredUsage } from "../domain/stored-usage.js"; import { compareText } from "../domain/text-order.js"; import { parsePositions, @@ -23,7 +18,6 @@ import { LeftOutAdapter } from "./left-out/left-out-adapter.js"; import { RepositoryAliasStoreAdapter } from "./legacy/repository-alias-store-adapter.js"; const DAILY = /^\d{4}-\d{2}-\d{2}\.jsonl$/; -const MONTHLY = /^\d{4}-\d{2}\.jsonl$/; const EXTENSION = ".jsonl"; const OFFSETS_FILE = "offsets.json"; const LOCK_FILE = ".lock"; @@ -38,17 +32,12 @@ function partitionOf(name: string): string { * snapshot of a call replaces the one held, and its time can fall in another day, so the * ledger is never appended to: a changed day is rewritten whole, by temporary file and * rename, and an unchanged one is not touched, so no reader ever meets a torn line. An ingest - * reads the days around its calls, plus every day of a month it splits, and writes only what - * changed, so its cost does not grow with the ledger; one storing a call with no request id - * reads every day (see `storeCalls`). Between reports, a call whose snapshots - * fell outside that window may sit in two days, and a crashed split leaves it in a month and a - * day: every reader folds at read, so it is still counted once. Once a report with no bound - * has settled the ledger, each call is on disk once and the bytes depend on what the ledger - * holds and not on how it got there. - * - * `.jsonl` is a month as an earlier version wrote it: still read, and split into its - * days, its own file removed after them, so a crash in between leaves each call in both and - * read once. + * reads the days around its calls and writes only what changed, so its cost does not grow with + * the ledger; one storing a call with no request id reads every day (see `storeCalls`). + * Between reports, a call whose snapshots fell outside that window may sit in two days: every + * reader folds at read, so it is still counted once. Once a report with no bound has settled + * the ledger, each call is on disk once and the bytes depend on what the ledger holds and not + * on how it got there. * * `offsets.json` is where each transcript was read up to; `left-out.txt` what ingests measured * and could not store (see `LeftOutAdapter`); `.lock` serialises the writers of all of them. */ @@ -80,9 +69,9 @@ export class UsageLedgerAdapter implements UsageLedger { // Every read applies the rules of now, so a call an earlier version keyed or named // otherwise is still one call of one repository, and its partition is rewritten. const ids = (await this.aliases.load())?.ids ?? new Map(); - for (const name of holds === undefined ? names : inWindow(names, holds)) { + for (const name of names) { const partition = partitionOf(name); - if (MONTHLY.test(name)) unsettled.add(partition); + if (holds !== undefined && !holds(partition)) continue; const text = (await readTextIfPresent(join(this.dir, name))) ?? ""; for (const line of text.split("\n")) { if (line === "") continue; @@ -106,20 +95,14 @@ export class UsageLedgerAdapter implements UsageLedger { async save(records: readonly StoredUsage[], partitions?: ReadonlySet): Promise { await this.storage.ensureDirectory(this.dir); - const named = (day: string): boolean => - partitions === undefined || partitions.has(day) || partitions.has(day.slice(0, 7)); + const named = (day: string): boolean => partitions === undefined || partitions.has(day); const held = new Map(); const kept = new Map>(); - const months: string[] = []; for (const name of await this.partitions()) { - const partition = partitionOf(name); - const monthly = MONTHLY.test(name); - // A month's lines that are not records go to its first day: their content is never guessed. - const day = monthly ? `${partition}-01` : partition; - if (monthly ? partitions !== undefined && !partitions.has(partition) : !named(day)) continue; + const day = partitionOf(name); + if (!named(day)) continue; const text = await readTextIfPresent(join(this.dir, name)); - if (monthly) months.push(name); - else held.set(day, text); + held.set(day, text); for (const line of (text ?? "").split("\n")) { if (line !== "" && parseStoredUsage(line) === null) { kept.set(day, (kept.get(day) ?? new Set()).add(line)); @@ -141,8 +124,8 @@ export class UsageLedgerAdapter implements UsageLedger { else if (held.get(day) !== content) await this.storage.replace(path, content); } // After the writes: a record whose better snapshot moved to another day can leave its old - // day empty, and until every day of a month is written, its monthly file holds the calls. - for (const path of [...emptied, ...months.map((name) => join(this.dir, name))]) await rm(path); + // day empty. + for (const path of emptied) await rm(path); } async positions(): Promise> { @@ -156,38 +139,10 @@ export class UsageLedgerAdapter implements UsageLedger { private async partitions(): Promise { try { - return (await readdir(this.dir)).filter((name) => DAILY.test(name) || MONTHLY.test(name)); + return (await readdir(this.dir)).filter((name) => DAILY.test(name)); } catch (error) { if (isErrnoException(error) && error.code === "ENOENT") return []; throw error; } } } - -/** Whether the test holds one of the calendar days of a `YYYY-MM` month; never, for a month - * no calendar has. */ -function holdsADayOf(month: string, holds: (day: string) => boolean): boolean { - if (Number.isNaN(Date.parse(`${month}-01T00:00:00.000Z`))) return false; - for (let day = `${month}-01`; day.startsWith(month); day = shiftDay(day, 1)) { - if (holds(day)) return true; - } - return false; -} - -/** The days the test holds, and every partition of a month it holds a day of that still has - * its monthly file: a half-finished split may hold a better snapshot in any day of that month. - * Each partition on disk is tested once, and a month at most once per calendar day. */ -function inWindow(names: readonly string[], holds: (day: string) => boolean): string[] { - const splitting = new Set( - names - .filter((name) => MONTHLY.test(name)) - .map(partitionOf) - .filter((month) => holdsADayOf(month, holds)) - ); - return names.filter((name) => { - const partition = partitionOf(name); - return MONTHLY.test(name) - ? splitting.has(partition) - : splitting.has(partition.slice(0, 7)) || holds(partition); - }); -} diff --git a/cli/tests/contexts/telemetry/application/ingest-usage-use-case.unit.test.ts b/cli/tests/contexts/telemetry/application/ingest-usage-use-case.unit.test.ts index 9c7a78dfb..b62854c84 100644 --- a/cli/tests/contexts/telemetry/application/ingest-usage-use-case.unit.test.ts +++ b/cli/tests/contexts/telemetry/application/ingest-usage-use-case.unit.test.ts @@ -373,15 +373,15 @@ describe("ingesting usage into the ledger", () => { expect([...(s.ledger.savedPartitions ?? [])]).toEqual(["2026-10-07"]); }); - it("finishes a monthly partition its window read even when every call was already held", async () => { + it("settles an unsettled day its window read even when every call was already held", async () => { const s = setup(); s.transcripts.files.set("/t/1.jsonl", [line("A", 10)]); await s.ingest.execute(); - s.ledger.unsettled.add("2026-10"); + s.ledger.unsettled.add("2026-10-07"); s.ledger.stored = new Map(); s.ledger.savedPartitions = undefined; expect(await s.ingest.execute()).toMatchObject({ added: 0, updated: 0 }); - expect([...(s.ledger.savedPartitions ?? [])]).toEqual(["2026-10"]); + expect([...(s.ledger.savedPartitions ?? [])]).toEqual(["2026-10-07"]); expect(s.ledger.records).toHaveLength(1); }); diff --git a/cli/tests/contexts/telemetry/application/ledger-upkeep.integration.test.ts b/cli/tests/contexts/telemetry/application/ledger-upkeep.integration.test.ts index fb480ce33..562b550f4 100644 --- a/cli/tests/contexts/telemetry/application/ledger-upkeep.integration.test.ts +++ b/cli/tests/contexts/telemetry/application/ledger-upkeep.integration.test.ts @@ -6,14 +6,12 @@ import { settleLedger, storeCalls, } from "../../../../src/contexts/telemetry/application/ledger-upkeep.js"; -import { buildReport } from "../../../../src/contexts/telemetry/domain/report/usage-report.js"; import { partitionByDay, type StoredUsage, } from "../../../../src/contexts/telemetry/domain/stored-usage.js"; import { UsageLedgerAdapter } from "../../../../src/contexts/telemetry/infrastructure/usage-ledger-adapter.js"; import { PrivateStorageAdapter } from "../../../../src/runtime/filesystem/private-storage-adapter.js"; -import { FailingStorage } from "../../../helpers/ports/failing-private-storage.js"; let root: string; @@ -62,16 +60,6 @@ function daysOf(records: readonly StoredUsage[]): Map { return days; } -/** A month as an earlier version wrote it: its days' lines, in order, in one file. */ -function monthsOf(records: readonly StoredUsage[]): Map { - const months = new Map(); - for (const [name, text] of daysOf(records)) { - const month = `${name.slice(0, 7)}.jsonl`; - months.set(month, (months.get(month) ?? "") + text); - } - return months; -} - async function ledgerAt(name: string, files: ReadonlyMap) { const dir = join(root, name); await mkdir(dir, { recursive: true }); @@ -82,68 +70,16 @@ async function ledgerAt(name: string, files: ReadonlyMap) { async function bytes(dir: string): Promise> { const found: Record = {}; for (const name of (await readdir(dir)).sort()) { - if (/^\d{4}-\d{2}(-\d{2})?\.jsonl$/.test(name)) { + if (/^\d{4}-\d{2}-\d{2}\.jsonl$/.test(name)) { found[name] = await readFile(join(dir, name), "utf8"); } } return found; } -const NO_FACTS = { declarations: [], carries: [], branches: new Map() }; - -async function totals(ledger: UsageLedgerAdapter) { - return buildReport((await ledger.load()).records, NO_FACTS, null, "total").totals; -} - const days = () => daysOf(CALLS); -const months = () => monthsOf(CALLS); -const septemberDays = () => [...days()].filter(([name]) => name.startsWith("2026-09")); - -/** Every disk state a split of September can be caught in, October still monthly. */ -const SPLIT_STATES: readonly (readonly [string, () => Map])[] = [ - ["the monthly file only", () => new Map(months())], - [ - "the monthly file and some of its days", - () => new Map([...months(), ...septemberDays().slice(0, 1)]), - ], - ["the monthly file and all of its days", () => new Map([...months(), ...septemberDays()])], - [ - "its days only", - () => - new Map([ - ...septemberDays(), - ...[...months()].filter(([name]) => name.startsWith("2026-10")), - ]), - ], -]; - -describe("migrating a ledger kept by month", () => { - it.each(SPLIT_STATES)("reports the same totals with %s", async (_state, files) => { - const before = await ledgerAt("before", months()); - const during = await ledgerAt("during", files()); - expect(await totals(during.ledger)).toEqual(await totals(before.ledger)); - }); - - it.each(SPLIT_STATES)("leaves only the days once settled, from %s", async (_state, files) => { - const { dir, ledger } = await ledgerAt("settling", files()); - const fresh = await ledgerAt("fresh", new Map()); - await fresh.ledger.save(CALLS); - const held = await totals(ledger); - await settleLedger(ledger); - expect(await bytes(dir)).toEqual(await bytes(fresh.dir)); - expect(await totals(ledger)).toEqual(held); - }); - - it.each(SPLIT_STATES)( - "leaves only the days once an ingest's window reaches both months, from %s", - async (_state, files) => { - const { dir, ledger } = await ledgerAt("ingesting", files()); - const result = await storeCalls(ledger, [stored("c", "2026-09-30T23:59:00.000Z", 30)]); - expect(result).toMatchObject({ added: 0, updated: 0 }); - expect(await bytes(dir)).toEqual(Object.fromEntries([...days()].sort())); - } - ); +describe("settling a ledger", () => { it("settles nothing, and writes nothing, on a ledger already in days", async () => { const { dir, ledger } = await ledgerAt("settled", days()); const before = await bytes(dir); @@ -170,40 +106,6 @@ describe("migrating a ledger kept by month", () => { describe("settling only a window", () => { /** Mid-September, so the window touches September and no day of October. */ const window = (day: string): boolean => day >= "2026-09-14" && day <= "2026-09-16"; - const october = () => [...months()].filter(([name]) => name.startsWith("2026-10")); - - async function linesOf(dir: string): Promise { - const files = Object.values(await bytes(dir)); - return files.flatMap((text) => text.split("\n").slice(0, -1)).length; - } - - it("splits a monthly file the window touches, loses and doubles no call, and leaves the rest alone", async () => { - const { dir, ledger } = await ledgerAt("bounded", months()); - const held = await totals(ledger); - await settleLedger(ledger, window); - const after = await bytes(dir); - expect(after).toEqual({ - ...Object.fromEntries(septemberDays()), - ...Object.fromEntries(october()), - }); - expect(await linesOf(dir)).toBe(CALLS.length); - expect(await totals(ledger)).toEqual(held); - }); - - it("gives the same figures after a crash mid-split, once settled again", async () => { - const { dir, ledger } = await ledgerAt("crashed", months()); - const held = await totals(ledger); - const crashing = new UsageLedgerAdapter(dir, new FailingStorage(2)); - await expect(settleLedger(crashing, window)).rejects.toThrow("disk full"); - expect(await totals(ledger)).toEqual(held); - await settleLedger(ledger, window); - expect(await totals(ledger)).toEqual(held); - expect(await linesOf(dir)).toBe(CALLS.length); - expect(await bytes(dir)).toEqual({ - ...Object.fromEntries(septemberDays()), - ...Object.fromEntries(october()), - }); - }); it("neither reads nor repairs a damaged day outside the window", async () => { const damaged = new Map(days()); diff --git a/cli/tests/contexts/telemetry/application/ledger-upkeep.property.integration.test.ts b/cli/tests/contexts/telemetry/application/ledger-upkeep.property.integration.test.ts index bca1a70c5..d3ddb80db 100644 --- a/cli/tests/contexts/telemetry/application/ledger-upkeep.property.integration.test.ts +++ b/cli/tests/contexts/telemetry/application/ledger-upkeep.property.integration.test.ts @@ -80,10 +80,11 @@ const scenario = fc.array(call, { minLength: 1, maxLength: 12 }).chain((calls) = { minLength: calls.length, maxLength: calls.length } ), batchOf: each(fc.nat(4)), - monthly: each(fc.boolean()), - // A call an earlier version kept that a transcript read again offers once more. + // A call the ledger already holds before the first batch. + held: each(fc.boolean()), + // A held call that a transcript read again offers once more. alsoIngested: each(fc.boolean()), - tornMonth: fc.boolean(), + tornDay: fc.boolean(), strayDay: fc.boolean(), // Lines a newer version wrote, or a hand edit left, in days of their own. foreign: fc.array( @@ -137,15 +138,13 @@ function linesIn(files: Record): string[] { return Object.values(files).flatMap((text) => text.split("\n").slice(0, -1)); } -/** What an earlier version would have written for these calls: one file per month. */ -function monthsOf(records: readonly StoredUsage[]): Map { - const months = new Map(); +/** The days an earlier ingest of these calls wrote. */ +function daysOf(records: readonly StoredUsage[]): Map { + const days = new Map(); for (const [day, inDay] of partitionByDay(foldUsage(records))) { - const month = `${day.slice(0, 7)}.jsonl`; - const text = `${inDay.map((record) => JSON.stringify(record)).join("\n")}\n`; - months.set(month, (months.get(month) ?? "") + text); + days.set(`${day}.jsonl`, `${inDay.map((record) => JSON.stringify(record)).join("\n")}\n`); } - return months; + return days; } async function freshLedger(files: ReadonlyMap = new Map()) { @@ -159,7 +158,7 @@ async function freshLedger(files: ReadonlyMap = new Map()) { async function bytes(dir: string): Promise> { const found: Record = {}; for (const name of (await readdir(dir)).sort()) { - if (/^\d{4}-\d{2}(-\d{2})?\.jsonl$/.test(name)) { + if (/^\d{4}-\d{2}-\d{2}\.jsonl$/.test(name)) { found[name] = await readFile(join(dir, name), "utf8"); } } @@ -167,7 +166,7 @@ async function bytes(dir: string): Promise> { } describe("the ledger's bytes, once a report has run", () => { - it("are those of one ingest of the same calls, whatever the order, grouping, starting format, damage and lines no reader knows, and no such line is ever lost", async () => { + it("are those of one ingest of the same calls, whatever the order, grouping, starting ledger, damage and lines no reader knows, and no such line is ever lost", async () => { await fc.assert( fc.asyncProperty( scenario, @@ -175,22 +174,22 @@ describe("the ledger's bytes, once a report has run", () => { calls, order, batchOf, - monthly, + held, alsoIngested, - tornMonth, + tornDay, strayDay, foreign, reports, }) => { - const earlier = calls.filter((_, index) => monthly[index]); - const files = monthsOf(earlier); + const earlier = calls.filter((_, index) => held[index]); + const files = daysOf(earlier); const kept = new Map(); const keep = (name: string, line: string) => kept.set(name, [...(kept.get(name) ?? []), line]); - const firstMonth = [...files.keys()][0]; - if (tornMonth && firstMonth !== undefined) { - files.set(firstMonth, `${files.get(firstMonth)}${TORN}\n`); - keep(`${firstMonth.slice(0, 7)}-01.jsonl`, TORN); + const firstDay = [...files.keys()][0]; + if (tornDay && firstDay !== undefined) { + files.set(firstDay, `${files.get(firstDay)}${TORN}\n`); + keep(firstDay, TORN); } if (strayDay) { files.set(STRAY_DAY, `${TORN}\n`); @@ -212,7 +211,7 @@ describe("the ledger's bytes, once a report has run", () => { }; const batches: StoredUsage[][] = [[], [], [], [], []]; for (const index of order) { - if (!monthly[index] || alsoIngested[index]) { + if (!held[index] || alsoIngested[index]) { batches[batchOf[index] as number]?.push(calls[index] as StoredUsage); } } diff --git a/cli/tests/contexts/telemetry/application/legacy/stored-usage-migration.property.integration.test.ts b/cli/tests/contexts/telemetry/application/legacy/stored-usage-migration.property.integration.test.ts index 942ef5585..c6c729d0d 100644 --- a/cli/tests/contexts/telemetry/application/legacy/stored-usage-migration.property.integration.test.ts +++ b/cli/tests/contexts/telemetry/application/legacy/stored-usage-migration.property.integration.test.ts @@ -143,8 +143,6 @@ const scenario = fc.record({ copies: fc.array(copy, { minLength: 1, maxLength: 24 }), /** How the earlier version's reads were cut: each copy read before goes into one of them. */ batches: fc.array(fc.nat(3), { minLength: 24, maxLength: 24 }), - /** Whether October is still in the earlier monthly layout. */ - monthly: fc.boolean(), /** Clones git cannot read, or not there (an unmounted volume), at the first run after the * upgrade, and back by the next. */ away: fc.array( @@ -178,8 +176,7 @@ async function writeEarlierStore(dir: string, bindings: string, s: Scenario): Pr }); for (const stored of foldUsage(read)) { const day = dayOf(stored.at) as string; - const name = s.monthly && day.startsWith("2026-10") ? day.slice(0, 7) : day; - await appendFile(join(dir, `${name}.jsonl`), `${JSON.stringify(stored)}\n`); + await appendFile(join(dir, `${day}.jsonl`), `${JSON.stringify(stored)}\n`); } } await writeRoots(dir, legacyRepositoryIdOf); @@ -343,7 +340,6 @@ describe("stored data from an earlier version, once migrated", () => { { call: 0, time: 0, output: 0, before: false }, ], batches: Array.from({ length: 24 }, () => 0), - monthly: false, away: [], }); }); diff --git a/cli/tests/contexts/telemetry/application/report/report-figures.property.integration.test.ts b/cli/tests/contexts/telemetry/application/report/report-figures.property.integration.test.ts index daf70a9d4..c406fa6c5 100644 --- a/cli/tests/contexts/telemetry/application/report/report-figures.property.integration.test.ts +++ b/cli/tests/contexts/telemetry/application/report/report-figures.property.integration.test.ts @@ -140,19 +140,17 @@ const scenario = fc.record({ }), { maxLength: 5 } ), - // Lines no version reads, already on disk, some twice, some in a month an earlier version - // kept in one file: settling keeps each once, in its day or its month's first day. + // Lines no version reads, already on disk, some twice: settling keeps each once, in its day. damaged: fc.array( fc.record({ day: fc.constantFrom(...DAYS), text: fc.constantFrom("x", "y"), - inMonth: fc.boolean(), }), { maxLength: 3 } ), - // Snapshots an earlier version already wrote, in their month's single file: both histories - // start from them, and the first read that reaches the month splits it into days. - inMonth: fc.array(fc.boolean(), { minLength: 40, maxLength: 40 }), + // Snapshots an earlier ingest already wrote, each in its day, unfolded: both histories start + // from them, and the first read that reaches a day settles it. + heldBefore: fc.array(fc.boolean(), { minLength: 40, maxLength: 40 }), request: fc.record({ axis: fc.constantFrom(...REPORT_AXES), period, filters }), // How another history reached the same data: each snapshot and left-out entry in a batch of // its own choosing, the batches in any order, and stores that bring nothing in between. @@ -193,19 +191,19 @@ function recordsOf(calls: Scenario["calls"]): StoredUsage[] { ); } -/** A ledger directory as an earlier version or a hand edit left it: records in their month's - * single file, and lines that are no record in a day's file or a month's. */ +/** A ledger directory as an earlier ingest or a hand edit left it: records in their day's file, + * unfolded, and lines that are no record. */ async function ledgerWith( - damaged: readonly { day: string; text: string; inMonth?: boolean }[], - monthly: readonly StoredUsage[] = [] + damaged: readonly { day: string; text: string }[], + earlier: readonly StoredUsage[] = [] ): Promise { ledgers += 1; const dir = join(root, `ledger-${ledgers}`); await mkdir(dir, { recursive: true }); const files = new Map(); const put = (name: string, line: string) => files.set(name, `${files.get(name) ?? ""}${line}\n`); - for (const record of monthly) put(record.at.slice(0, 7), JSON.stringify(record)); - for (const { day, text, inMonth } of damaged) put(inMonth ? day.slice(0, 7) : day, text); + for (const record of earlier) put(record.at.slice(0, 10), JSON.stringify(record)); + for (const { day, text } of damaged) put(day, text); for (const [name, text] of files) await writeFile(join(dir, `${name}.jsonl`), text); return new UsageLedgerAdapter(dir, new PrivateStorageAdapter()); } @@ -293,19 +291,20 @@ describe("the figures of a report", () => { await fc.assert( fc.asyncProperty(scenario, async (held) => { const all = recordsOf(held.calls); - const monthly = all.filter((_, n) => held.inMonth[n % held.inMonth.length]); - const records = all.filter((_, n) => !held.inMonth[n % held.inMonth.length]); + const wasHeld = (n: number) => held.heldBefore[n % held.heldBefore.length]; + const earlier = all.filter((_, n) => wasHeld(n)); + const records = all.filter((_, n) => !wasHeld(n)); const leftOut: LeftOutEntry[] = held.leftOut; // One history: every call and left-out entry stored at once, reported in UTC. - const once = await ledgerWith(held.damaged, monthly); + const once = await ledgerWith(held.damaged, earlier); await once.exclusively(() => storeCalls(once, records)); await once.leftOut.add(leftOut); const reference = await report(held, once, held.request, held.runs[0], "UTC"); // Another: in batches, in another order, with stores that bring nothing, stored with // the clock in another zone. - const batched = await ledgerWith(held.damaged, monthly); + const batched = await ledgerWith(held.damaged, earlier); const { batchOf, order, noOps } = held.history; const nth = (n: number) => batchOf[n % batchOf.length] as number; const [first, second] = held.zones; diff --git a/cli/tests/contexts/telemetry/application/report/report-window.property.integration.test.ts b/cli/tests/contexts/telemetry/application/report/report-window.property.integration.test.ts index e6b19cfef..4760a076d 100644 --- a/cli/tests/contexts/telemetry/application/report/report-window.property.integration.test.ts +++ b/cli/tests/contexts/telemetry/application/report/report-window.property.integration.test.ts @@ -84,9 +84,6 @@ const copy = fc.record({ model: fc.constantFrom("m-1", "m-2"), agent: fc.constantFrom(...USAGE_AGENTS), repository: fc.constantFrom("repo-1", "repo-2"), - // Where this copy sits on disk: its day, its month as an earlier version wrote it, or both, - // as a split that crashed half-way leaves it. - placed: fc.constantFrom("day", "month", "both"), }); const call = fc.record({ @@ -148,7 +145,7 @@ function filesOf(calls: readonly Call[]): Map { const put = (name: string, record: StoredUsage) => files.set(name, `${files.get(name) ?? ""}${JSON.stringify(record)}\n`); calls.forEach(({ anchor, tool, copies }, index) => { - for (const { offset, output, session, model, agent, repository, placed } of copies) { + for (const { offset, output, session, model, agent, repository } of copies) { const at = new Date((ANCHORS[anchor] as number) + offset).toISOString(); const record: StoredUsage = { tool, @@ -168,8 +165,7 @@ function filesOf(calls: readonly Call[]): Map { git_branch: null, repository_id: repository, }; - if (placed !== "month") put(`${at.slice(0, 10)}.jsonl`, record); - if (placed !== "day") put(`${at.slice(0, 7)}.jsonl`, record); + put(`${at.slice(0, 10)}.jsonl`, record); } }); return files; diff --git a/cli/tests/contexts/telemetry/infrastructure/forget/measurement-erasure-adapter.integration.test.ts b/cli/tests/contexts/telemetry/infrastructure/forget/measurement-erasure-adapter.integration.test.ts index 1ba4e5b5e..5f10a5660 100644 --- a/cli/tests/contexts/telemetry/infrastructure/forget/measurement-erasure-adapter.integration.test.ts +++ b/cli/tests/contexts/telemetry/infrastructure/forget/measurement-erasure-adapter.integration.test.ts @@ -79,9 +79,9 @@ describe("what measurement keeps", () => { ]); }); - it("removes the ledger's daily and monthly partitions as the ledger, never as an earlier day file", async () => { + it("removes the ledger's day files as the ledger, never as an earlier day file", async () => { write(join(telemetry, "ledger", "2026-10-07.jsonl")); - write(join(telemetry, "ledger", "2026-09.jsonl")); + write(join(telemetry, "ledger", "2026-09-30.jsonl")); write(join(telemetry, "ledger", "offsets.json")); write(join(sink, "2026-10-05.jsonl")); const found = await erasure().inventory(); diff --git a/cli/tests/contexts/telemetry/infrastructure/usage-ledger-adapter.integration.test.ts b/cli/tests/contexts/telemetry/infrastructure/usage-ledger-adapter.integration.test.ts index ba0cb4a02..b9b5ed28d 100644 --- a/cli/tests/contexts/telemetry/infrastructure/usage-ledger-adapter.integration.test.ts +++ b/cli/tests/contexts/telemetry/infrastructure/usage-ledger-adapter.integration.test.ts @@ -5,7 +5,6 @@ import { afterEach, beforeEach, describe, expect, it } from "vitest"; import type { StoredUsage } from "../../../../src/contexts/telemetry/domain/stored-usage.js"; import { UsageLedgerAdapter } from "../../../../src/contexts/telemetry/infrastructure/usage-ledger-adapter.js"; import { PrivateStorageAdapter } from "../../../../src/runtime/filesystem/private-storage-adapter.js"; -import { FailingStorage } from "../../../helpers/ports/failing-private-storage.js"; let root: string; let dir: string; @@ -42,7 +41,7 @@ function stored(overrides: Partial = {}): StoredUsage { }; } -const PARTITION_NAME = /^\d{4}-\d{2}(-\d{2})?\.jsonl$/; +const PARTITION_NAME = /^\d{4}-\d{2}-\d{2}\.jsonl$/; const partitions = async (): Promise => (await readdir(dir)).filter((name) => PARTITION_NAME.test(name)).sort(); @@ -161,10 +160,11 @@ describe("the ledger on disk", () => { expect(records.map((r) => r.output)).toEqual([90]); }); - it("reads only partition files, and leaves the offsets, roots, consents and lock beside them alone", async () => { + it("reads only day files, and leaves a month's name, the offsets, roots, consents and lock alone", async () => { await ledger.savePositions(new Map([["/t", { offset: 1, size: 1, identity: "i" }]])); const beside = [ "x2026-10-07.jsonl", + "2026-10.jsonl", "x2026-10.jsonl", "2026-10-07.jsonl.bak", "26-10-07.jsonl", @@ -323,62 +323,11 @@ describe("reading a window of days", () => { expect(loaded.skippedLines).toBe(0); }); - it("reads a monthly file the window touches by one day, with every day of its month", async () => { - const early = stored({ key: "early", at: "2026-02-01T10:00:00Z" }); - const late = stored({ key: "late", at: "2026-02-27T10:00:00Z" }); - const daily = stored({ key: "daily", at: "2026-02-03T10:00:00Z" }); - await writePartition("2026-02.jsonl", lines([early, late])); - await writePartition("2026-02-03.jsonl", lines([daily])); - await writePartition( - "2026-03-01.jsonl", - lines([stored({ key: "march", at: "2026-03-01T10:00:00Z" })]) - ); - const loaded = await ledger.load((day) => day >= "2026-02-28" && day <= "2026-03-01"); - expect(loaded.records.map((r) => r.key)).toEqual(["daily", "early", "late", "march"]); - expect([...loaded.unsettled]).toEqual(["2026-02"]); - }); - - it("does not read a monthly file the window touches on none of its days", async () => { - await writePartition( - "2026-02.jsonl", - `${lines([stored({ at: "2026-02-28T10:00:00Z" })])}{torn\n` - ); - await writePartition("2026-02-28.jsonl", "{torn\n"); - expect(await ledger.load((day) => day >= "2026-03-01")).toEqual(NOTHING); - expect(await ledger.load((day) => day <= "2026-01-31")).toEqual(NOTHING); - }); - - it("passes over a monthly name no calendar holds, as it did before windows were tests", async () => { - await ledger.save([stored()]); - await writePartition("2026-13.jsonl", lines([stored({ key: "never" })])); - expect((await ledger.load(only("2026-10-07"))).records).toEqual([stored()]); - }); - it("reads nothing for an empty window", async () => { await ledger.save([stored()]); expect(await ledger.load(() => false)).toEqual(NOTHING); }); - it("reads a monthly partition, and the daily ones of its month, when a day of the window falls in it", async () => { - const early = stored({ key: "early", at: "2026-09-10T10:00:00Z" }); - const late = stored({ key: "late", at: "2026-09-28T10:00:00Z" }); - const daily = stored({ key: "daily", at: "2026-09-25T10:00:00Z" }); - await writePartition("2026-09.jsonl", lines([early, late])); - await writePartition("2026-09-25.jsonl", lines([daily])); - for (const loaded of [await ledger.load(), await ledger.load(only("2026-09-10"))]) { - expect(loaded.records.map((r) => r.key)).toEqual(["daily", "early", "late"]); - expect([...loaded.unsettled]).toEqual(["2026-09"]); - } - expect(await ledger.load(only("2026-10-07"))).toEqual(NOTHING); - }); - - it("reads a monthly partition its window reaches by the day before", async () => { - await writePartition("2026-09.jsonl", lines([stored({ at: "2026-09-30T10:00:00Z" })])); - const loaded = await ledger.load(only("2026-09-30", "2026-10-01")); - expect(loaded.records).toHaveLength(1); - expect([...loaded.unsettled]).toEqual(["2026-09"]); - }); - it("finds a call held in two days of its window", async () => { await writePartition( "2026-10-06.jsonl", @@ -391,105 +340,6 @@ describe("reading a window of days", () => { }); }); -describe("splitting a monthly partition into its days", () => { - const september = [ - stored({ key: "a", at: "2026-09-01T00:00:00Z" }), - stored({ key: "b", at: "2026-09-15T12:00:00Z" }), - stored({ key: "c", at: "2026-09-30T23:59:00Z" }), - ]; - - it("writes each day of the month, then removes the monthly file, in the bytes a fresh ledger has", async () => { - await writePartition("2026-09.jsonl", lines(september)); - await writePartition( - "2026-09-20.jsonl", - lines([stored({ key: "gone", at: "2026-09-20T00:00:00Z" })]) - ); - await ledger.save(september, new Set(["2026-09"])); - const fresh = new UsageLedgerAdapter(join(root, "fresh"), new PrivateStorageAdapter()); - await fresh.save(september); - const names = ["2026-09-01.jsonl", "2026-09-15.jsonl", "2026-09-30.jsonl"]; - expect(await partitions()).toEqual(names); - for (const name of names) { - expect(await readFile(join(dir, name), "utf8")).toBe( - await readFile(join(root, "fresh", name), "utf8") - ); - } - }); - - it("writes only the days of the month it is told", async () => { - await writePartition("2026-09.jsonl", lines(september)); - await ledger.save( - [...september, stored({ key: "d", at: "2026-10-01T00:00:00Z" })], - new Set(["2026-09"]) - ); - expect(await partitions()).not.toContain("2026-10-01.jsonl"); - }); - - it("removes the monthly file only after its days are written", async () => { - await writePartition("2026-09.jsonl", lines(september)); - const crashing = new UsageLedgerAdapter(dir, new FailingStorage(2)); - await expect(crashing.save(september, new Set(["2026-09"]))).rejects.toThrow("disk full"); - expect(await partitions()).toContain("2026-09.jsonl"); - const { records } = await ledger.load(); - expect(records.map((r) => r.key)).toEqual(["a", "b", "c"]); - }); - - it("removes every monthly file when told to hold exactly some records", async () => { - await writePartition("2026-09.jsonl", lines(september)); - await ledger.save(september); - expect(await partitions()).toEqual([ - "2026-09-01.jsonl", - "2026-09-15.jsonl", - "2026-09-30.jsonl", - ]); - }); - - it("keeps the better snapshot a half-finished split already wrote, and finishes it", async () => { - const worse = stored({ key: "a", output: 1, at: "2026-09-10T10:00:00Z" }); - const better = stored({ key: "a", output: 90, at: "2026-09-10T10:00:01Z" }); - const other = stored({ key: "b", at: "2026-09-20T10:00:00Z" }); - await writePartition("2026-09.jsonl", lines([worse, other])); - await writePartition("2026-09-10.jsonl", lines([better])); - const loaded = await ledger.load(only("2026-09-20")); - expect(loaded.records).toEqual([better, other]); - expect([...loaded.unsettled].sort()).toEqual(["2026-09", "2026-09-10"]); - await ledger.save(loaded.records, loaded.unsettled); - expect(await partitions()).toEqual(["2026-09-10.jsonl", "2026-09-20.jsonl"]); - expect(await ledger.load()).toEqual({ ...NOTHING, records: [better, other] }); - }); - - it("counts a line of a monthly file that is not a record, and keeps it in the first day of its month", async () => { - await writePartition("2026-09.jsonl", `${lines(september)}{torn\n`); - const loaded = await ledger.load(); - expect(loaded.skippedLines).toBe(1); - expect([...loaded.damaged]).toEqual(["2026-09"]); - await ledger.save(loaded.records, new Set([...loaded.damaged, ...loaded.unsettled])); - expect(await partitions()).toEqual([ - "2026-09-01.jsonl", - "2026-09-15.jsonl", - "2026-09-30.jsonl", - ]); - expect(await readFile(join(dir, "2026-09-01.jsonl"), "utf8")).toBe( - `${lines(september.slice(0, 1))}{torn\n` - ); - expect((await ledger.load()).skippedLines).toBe(1); - }); - - it("keeps a monthly file's line once, after a crash between its first day and the file's removal", async () => { - await writePartition("2026-09.jsonl", `${lines(september)}{torn\n`); - const crashing = new UsageLedgerAdapter(dir, new FailingStorage(2)); - await expect(crashing.save(september, new Set(["2026-09"]))).rejects.toThrow("disk full"); - expect(await readFile(join(dir, "2026-09-01.jsonl"), "utf8")).toContain("{torn"); - expect(await partitions()).toContain("2026-09.jsonl"); - const loaded = await ledger.load(); - await ledger.save(loaded.records, new Set([...loaded.damaged, ...loaded.unsettled])); - expect(await readFile(join(dir, "2026-09-01.jsonl"), "utf8")).toBe( - `${lines(september.slice(0, 1))}{torn\n` - ); - expect(await partitions()).not.toContain("2026-09.jsonl"); - }); -}); - describe("the positions of the transcripts read", () => { it("round-trip", async () => { const positions = new Map([ diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 4e508fe37..73399f5bf 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -115,7 +115,7 @@ Dependency-free Node scripts declared in `hooks/hooks.json`; `node` must be on ` | Plugin | Event | Runs | Purpose | | ---------------- | --------------------------------------- | ------------------------ | ----------------------------------------------------------- | | `aidd-context` | `SessionStart` | `hooks/update_memory.js` | Refresh the project memory block in the AI context files | -| `aidd-telemetry` | `SessionStart` · `UserPromptSubmit` | `hooks/prompt-gate.cjs` | Hold prompts on an undeclared branch until a task is declared, answer it without the model, keep it across `/clear`, catch the ledger up | +| `aidd-telemetry` | `SessionStart` · `UserPromptSubmit` | `hooks/session-start.cjs` · `hooks/catch-up.cjs` · `hooks/prompt-gate.cjs` | At session start, record which session a process is on and keep its task across `/clear`, and catch the ledger up (async); on a prompt, hold it on an undeclared branch until a task is declared, and answer the declaration without the model | ## Portability diff --git a/plugins/aidd-telemetry/CATALOG.md b/plugins/aidd-telemetry/CATALOG.md index 76172d46a..b96752897 100644 --- a/plugins/aidd-telemetry/CATALOG.md +++ b/plugins/aidd-telemetry/CATALOG.md @@ -45,5 +45,5 @@ Auto-generated index of skills, agents, references and assets shipped by the `ai | `actions` | [answer.md](skills/01-usage/actions/answer.md) | - | | `actions` | [frame.md](skills/01-usage/actions/frame.md) | - | | `references` | [envelope.md](skills/01-usage/references/envelope.md) | - | -| `-` | [SKILL.md](skills/01-usage/SKILL.md) | `Answer what a period or one piece of work consumed in tokens, per day, model, task, ticket, session, repository or person, from the local measurement. Use when the user asks how many tokens were spent, where the effort went, or what a task or ticket consumed. Not for turning measurement on or off.` | +| `-` | [SKILL.md](skills/01-usage/SKILL.md) | `Answer what a period or one piece of work consumed in tokens, per day, model, agent, task, ticket, session, repository or person, from the local measurement. Use when the user asks how many tokens were spent, where the effort went, or what a task or ticket consumed. Not for turning measurement on or off.` | diff --git a/plugins/aidd-telemetry/skills/01-usage/SKILL.md b/plugins/aidd-telemetry/skills/01-usage/SKILL.md index 627c1e31c..ade451464 100644 --- a/plugins/aidd-telemetry/skills/01-usage/SKILL.md +++ b/plugins/aidd-telemetry/skills/01-usage/SKILL.md @@ -1,6 +1,6 @@ --- name: 01-usage -description: Answer what a period or one piece of work consumed in tokens, per day, model, task, ticket, session, repository or person, from the local measurement. Use when the user asks how many tokens were spent, where the effort went, or what a task or ticket consumed. Not for turning measurement on or off. +description: Answer what a period or one piece of work consumed in tokens, per day, model, agent, task, ticket, session, repository or person, from the local measurement. Use when the user asks how many tokens were spent, where the effort went, or what a task or ticket consumed. Not for turning measurement on or off. argument-hint: question | period --- diff --git a/plugins/aidd-telemetry/skills/01-usage/references/envelope.md b/plugins/aidd-telemetry/skills/01-usage/references/envelope.md index 6cedb47ea..2dee16d71 100644 --- a/plugins/aidd-telemetry/skills/01-usage/references/envelope.md +++ b/plugins/aidd-telemetry/skills/01-usage/references/envelope.md @@ -15,8 +15,8 @@ A field may be added to version `2`: a figure outside `run`, a fact of the read | `version` | `2`. Any other value is a shape not known here | | `refused` | Only present, with `version`, when measurement is refused by the environment; nothing was read | | `period` | `from` and `to`, UTC days, `null` for open: resolved from the flags, `--days` against today's UTC day. The figures depend on these days, not on the flags | -| `axis` | What `rows` split by | -| `filters` | What the report was narrowed to: `task`, `ticket`, `person`, `model`, `agent`, `session`, `repository`, `absent`, `unattributed`. `null` when not given, else the value as resolved: a full id for a start or `.`, the axis for `absent`, the reason or `any` for `unattributed` | +| `axis` | What `rows` split by: `total`, `person`, `session`, `model`, `agent`, `day`, `repository`, `task` or `ticket` | +| `filters` | What the report was narrowed to: `task`, `ticket`, `person`, `model`, `agent`, `session`, `repository`, `absent`, `unattributed`, in that order. `null` when not given, else the value as resolved: a full id for a start or `.`, the axis for `absent`, the reason or `any` for `unattributed` | | `rows` | One per value of the axis, of the calls the filters keep; every axis adds up to the same `totals` | | `totals` | The calls of the period the filters keep; `totals.total.unknown_records` is how many had at least one counter unknown | | `coverage` | What the figures rest on | @@ -57,15 +57,17 @@ A field may be added to version `2`: a figure outside `run`, a fact of the read | `not_stored` | Calls this run read and left out, by reason: `outside-repo`, `never-seen-alive`, `no-consent`, `consent-closed`, `unreadable-consent`, `no-cwd`, `undated` | | `consent_log_damaged` | `true` while `ledger/consents.jsonl` cannot be trusted and nothing is stored | | `oldest_transcript_at` | When the oldest session file still on disk was last written, `null` when there is none | -| `skipped_ledger_lines` | Lines that were not records, kept as found (a newer version may have written them), in the stored days this run read before tidying them: with a period, those days and the day on each side, plus every day of a month still kept in one file | +| `skipped_ledger_lines` | Lines that were not records, kept as found (a newer version may have written them), in the stored days this run read before tidying them: with a period, those days and the day on each side | ## When there is nothing +`run.not_stored` and `run.files_read` describe only the read that printed the report. Except under `unreadable-consent`, a call one read left out is never read again: every later report shows `0` for it, so only the first report's `run` says why. `aidd telemetry status`, run in the clone, says whether it measures now. + | Sign | Likely reason | | --- | --- | -| `records` of `0` and `run.not_stored.no-consent` above `0` | The clone has not opted in, or the calls were made before it did or while it was off: measurement starts at `aidd telemetry on` and reads nothing back | +| `coverage.records` of `0` and `run.not_stored.no-consent` above `0` | The clone has not opted in, or the calls were made before it did or while it was off: measurement starts at `aidd telemetry on` and reads nothing back | | `run.not_stored.consent-closed` above `0` | The clone opted in once and its consent was closed: `aidd telemetry on` in that clone measures again, from then on | | `run.not_stored.unreadable-consent` above `0` | A clone's git config or git directory cannot be read: nothing is stored for it until it can. For a clone that had opted in, its calls are then stored, read again by the next run; for any other directory, they are not | | `run.consent_log_damaged` is `true` | A line of the consent log `ledger/consents.jsonl` is not an event, so nothing is stored for any clone. Recovery is `aidd telemetry forget --yes`, then `aidd telemetry on` in each clone: nothing from before is stored. Do not edit the log by hand | | `totals.records` of `0` and `coverage.records` above `0`, with a filter given | No call of the period matches the filters: a value no call has, or a start of a session or repository id that names none, printed as typed in `filters` | -| `records` of `0` and `run.files_read` of `0` | No clone on this machine has opted in, so no session file is read (`aidd telemetry status` says so in the clone); or no Claude Code session file was written since the first `aidd telemetry on` | +| `coverage.records` of `0`, `run.files_read` of `0` and every `run.not_stored` at `0` | Nothing new was read: no clone on this machine has opted in, so no session file is read (`aidd telemetry status` says so in the clone); no Claude Code session file was written since the first `aidd telemetry on`; or an earlier report already read the calls and left them out, and only its `run` said why | diff --git a/scripts/__tests__/architecture-doc-matches-the-tree.test.js b/scripts/__tests__/architecture-doc-matches-the-tree.test.js index 482b7564f..5167d3faa 100644 --- a/scripts/__tests__/architecture-doc-matches-the-tree.test.js +++ b/scripts/__tests__/architecture-doc-matches-the-tree.test.js @@ -126,16 +126,25 @@ test("the bundled hooks table names the events each plugin actually declares", ( } }); -test("the bundled hooks table names a script that exists", () => { - // Without this the table can point at a deleted hook and still read as current. +/** The scripts a plugin's `hooks/hooks.json` runs, relative to the plugin. */ +function declaredHookScripts(plugin) { + const manifest = fs.readFileSync(path.join(ROOT, "plugins", plugin, "hooks/hooks.json"), "utf8"); + return [...new Set([...manifest.matchAll(/\$\{CLAUDE_PLUGIN_ROOT\}\/([^\s"]+)/g)].map((m) => m[1]))].sort(); +} + +test("the bundled hooks table names every script a plugin runs, each one existing", () => { + // Without this the table can point at a deleted hook, or leave one out, and still read as current. const table = hooksTable(); const plugin = column(table, "Plugin"); const runs = column(table, "Runs"); for (const cells of table.rows) { const name = onlyBacktickedName(plugin(cells), "hooks table"); - const script = onlyBacktickedName(runs(cells), `hooks table, ${name} Runs cell`); - const full = path.join(ROOT, "plugins", name, script); - assert.ok(fs.existsSync(full), `plugins/${name}/${script} is named in ARCHITECTURE.md and does not exist`); + const scripts = backticked(runs(cells)).sort(); + assert.deepEqual(scripts, declaredHookScripts(name), `${name}: the Runs cell and plugins/${name}/hooks/hooks.json disagree`); + for (const script of scripts) { + const full = path.join(ROOT, "plugins", name, script); + assert.ok(fs.existsSync(full), `plugins/${name}/${script} is named in ARCHITECTURE.md and does not exist`); + } } }); diff --git a/scripts/__tests__/no-local-paths.test.js b/scripts/__tests__/no-local-paths.test.js index 16cd4e10e..a3501913b 100644 --- a/scripts/__tests__/no-local-paths.test.js +++ b/scripts/__tests__/no-local-paths.test.js @@ -106,12 +106,19 @@ describe("findLocalPaths", () => { ]); }); - it("reads every session array in a text, not only the first", () => { - const text = [`"threadIds": ["${SESSION}"],`, `"sessionIds": [],`, `"session_id": ["${SESSION_13}"]`].join("\n"); + it("reads every session array in a text, not only the first, whatever the case of its ids", () => { + const text = [ + `"threadIds": ["${SESSION}"],`, + `"sessionIds": [],`, + `"session_id": ["${SESSION_13}"]`, + `"threadIds": [`, + ` "${SESSION.toUpperCase()}"`, + "]", + ].join("\n"); assert.deepEqual( findLocalPaths(text).map(({ line }) => line), - [1, 3] + [1, 3, 5] ); }); @@ -119,22 +126,16 @@ describe("findLocalPaths", () => { assert.deepEqual(findLocalPaths(`"threadIds": [\n"${SESSION}"]`), [{ line: 2, label: "session id" }]); }); - it("stops at the first unclosed session key instead of rescanning the text for each", () => { + it("stops at the first unclosed session key instead of rescanning the text for each", (t) => { const unclosed = Array.from({ length: 1000 }, () => '"threadIds": [').join("\n"); // Counted, not timed: a time bound is flaky on a shared runner, a count of scans is not. - const indexOf = String.prototype.indexOf; - let scans = 0; - String.prototype.indexOf = function (search, ...rest) { - if (search === "]") scans++; - return indexOf.call(this, search, ...rest); - }; + const indexOf = t.mock.method(String.prototype, "indexOf"); - try { - assert.deepEqual(findLocalPaths(unclosed), []); - } finally { - String.prototype.indexOf = indexOf; - } - assert.equal(scans, 1); + const found = findLocalPaths(unclosed); + indexOf.mock.restore(); + + assert.deepEqual(found, []); + assert.equal(indexOf.mock.calls.filter((call) => call.arguments[0] === "]").length, 1); }); it("reads a key inside an array once, as part of that array", () => {