fix(telemetry): nothing measured is lost silently

[blafourcade]

🎯 What & why

This PR closes two gaps (M1 and M2 from the R1 telemetry audit) where telemetry lost data without saying so.

• M1. The plugin's catch-up hook runs aidd telemetry ingest --quiet. What that ingest measured but could not store was counted for its own run only, then the offsets moved on, so a report run later showed 0.
• M2. Rewriting a ledger partition deleted every line the reader did not recognise. An older CLI could therefore erase a newer CLI's records.

🛠️ How it works

• M2: unknown lines are kept. usage-ledger-adapter.ts:96 save keeps every non-empty line that is not a record.
  • The line stays in its own day, after the records, sorted, and each line once. A monthly file's lines go to the first day of that month.
  • A day holding only such lines is never removed.
  • skipped_ledger_lines counts these lines on every read.
  • Why keep them in the day and not in a quarantine file: a newer CLI finds its lines where it wrote them.
• M1: only what was measured is kept, and only as a hash. Rules are in usage-contract.md:81; the logic is at ingest-usage-use-case.ts:109.
  • A call excluded by consent or scope is not measured: it is counted by the run that read it and nothing of it is written. This covers a clone that never opted in, work after off, calls outside any repository, and directories never seen alive.
  • Two kinds of measured-but-unstorable data are kept, both only for a clone that consents:
    • an undated call;
    • a transcript line that is no usage line but names that clone's directory.
  • They go to ledger/left-out.txt as <reason> <32-hex sha-256>, one line each (41 or 46 bytes), each once, and never as a key or text in clear.
  • The store is its own small adapter (left-out-adapter.ts); the ledger only exposes it as leftOut.
• A call whose consent can't be read yet is read again. Examples: git refuses the .git once, or the config doesn't parse.
  • unreadable-consent is not a refusal when a clone that roots.json remembers at that directory consented at the call's time: that clone may consent again once it can be read. Undated calls and unrecognised lines follow the same rule.
  • Anything else moves on, and only that run reports it: a directory never seen readable, a repository that never opted in, or a call made while the clone did not consent. Once readable, those calls could never be stored, so a hold would never end. The same applies to a damaged consent log, which stays damaged until forget removes it.
  • Ingest keeps the position of each transcript that holds such a call from before that read (ingest-usage-use-case.ts:114). The next ingest, including the one every report runs, reads it again.
  • The call is stored once its consent can be read. Until then, every run reports it. Nothing is written to disk meanwhile.
• Envelope. It stays version: 1, with one field added: coverage.left_out: {undated, unrecognised}. It counts everything kept by every ingest so far, so it has the same value in every report, bounded or not. files_read, unrecognised_shapes and not_stored keep their meaning: the current run's.
• Bundle budget. Telemetry code was first trimmed by 1.7 KB, in its own refactor commits. The budget was then raised in its own commit, chore(cli): raise the bundle budget to 709 KB, as the policy in check-bundle-size.mjs requires, and recorded in its registry. Measured size: 712,014 B (695.3 KB), 2.0% headroom.

🧪 How to verify

• cd cli && pnpm test: Tests 6721 passed | 1 skipped (6722)
• node --test 'scripts/__tests__/**/*.test.js': pass 460, fail 0
• pnpm test:mutation:telemetry: score 96.6 (floor 96). The survivors on changed lines are explained in evidence.md, which also records the red runs, the hand mutations, smoke, and the token end-to-end check in both modes.
• Properties (fast-check):
  • left-out.property: any sequence of ingests, re-reads and offset resets, including while a clone that consents cannot be read, gives the same bytes and envelope as one ingest. left_out matches an independent oracle, is the same bounded or unbounded, and is never written for a clone that never opted in.
  • report-window.property: with left-out.txt present, a bounded report equals the full one filtered.
  • ledger-upkeep.property: byte identity holds with unknown lines present, and none is ever lost.
• End to end: telemetry-journey.e2e checks two things.
  • A session only in a clone that never opted in leaves no left-out.txt.
  • A report run after the catch-up hook shows left_out.

⚠️ Heads-up

• left_out is cumulative. It counts everything every ingest has kept so far, and does not reset per run or per period, so do not sum it across reports. not_stored and unrecognised_shapes still describe the current run only.
• ingest output changed. Counts are grouped by thousands, the snapshot line comes first, and each line's output level comes from its type.
• Gone worktrees are a known loss. A call from a directory that was gone before any ingest saw it alive (never-seen-alive, for example a worktree created and removed between two ingests) is reported only by the run that read it, and the hook's run prints nothing. Its consent is unknown, so nothing may be kept, and the directory never comes back, so re-reading it would never end. A claude --worktree session's start sees its directory alive; a subagent's worktree has no such recovery. A SubagentStart or SubagentStop hook might cover it; that is unverified and out of scope.
• roots.json may name a non-opted clone. Resolving the directory of an undated call or an unrecognised line records the directory in roots.json when it is alive, in any clone. Dated calls already did this before this PR. It records a directory, never a call. The guarantee "a clone that never opted in leaves nothing" therefore applies to left-out.txt, not to roots.json.
• Cost of an unreadable clone. While a clone that consents stays unreadable, every ingest re-reads the part of its transcripts read since it became unreadable, until the clone is repaired.
• Run-only wording. Report lines that describe only the current run now say "by this read".
• Copy assumption. A call is assumed to have its copies all dated or all undated. The contract states this, and the property's generator respects it.
• Undated calls and consent. An undated call is judged by its clone's consent at the time of the read, because it has no time of its own.
• Lines that name no directory. Such a line cannot be tied to a clone, so only the run that read it reports it.
• cli-v5.5.0 still deletes ledger lines it does not recognise.
• Review. Checker rounds 1 to 3 are recorded in review.md, and every finding from all three is addressed in this push. No checker has reviewed the result yet.
• Unrelated, not fixed. forget pluralises "repository" as "repositorys".

🔗 Linked issue

None. Found by the R1 telemetry audit (M1, M2).

✅ I certify

• I DO CERTIFY I READ EACH LINE OF THE PULL REQUEST BECAUSE I AM A SOFTWARE ENGINEER, NOT A AI PUPPY.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VWNxk63AGKkqE8HRqHLjGp