Skip to content

History writes are synchronous SQLite writes on the sample path #39

Description

@bmethod

Source: measured while giving src/store/lane-series.test.ts a timeout
margin, 2026-09-09. Re-audited 2026-09-10 against origin/issue-24-design at
685e3e8.

Reached by: run the full test suite on a loaded machine. The reopened-history
case takes 0.34 seconds alone and about 4.7 seconds inside the suite, where files
run concurrently and compete for the same disk.

History writes are synchronous SQLite writes on the sample path, and they contend
for the disk with the very workload vsys is watching. 130 samples can take fifteen
times their idle cost. Nothing pins what a single write costs.

A machine running dozens of compiling agents is the machine this program is for.
The sample path competing with that workload for the disk is the ordinary case,
not an edge one.

This surfaced as a test that failed on timing rather than on what it asserts,
three times across three branches, and never reproduced when run alone. That test
now carries a 30-second margin at src/store/lane-series.test.ts:75, which is
correct and which also means this margin is what will absorb the evidence if the
write cost grows. That is the reason to measure it somewhere else.

What already exists, and what it does not measure

scripts/bench-history.ts fills a full retention window — 50 scopes, 40 pids
each — calling history.add(snapshot) at :87, and prints elapsedMs and
rssBytes for the whole fill at :106-119. It answers whether a full window
fits in memory and replays exactly. It does not report what one write costs, it
runs on an idle machine only, and scripts/ci.py has no reference to it, so
nothing runs it.

The work is to extend that script rather than to start one.

Done when

  • scripts/bench-history.ts states what one history write costs, and what a
    sample's worth of writes costs, on an idle machine and under load.
  • A stated budget for the sample path, with a test that fails when a write
    exceeds it — so a regression shows up as a budget failure rather than as an
    unrelated test timing out.

Context

  • Location: src/store/history.ts (add at :206, whose SQLite insert and
    retention delete run at :214-220), src/store/archive.ts, and
    scripts/bench-history.ts.
  • Related: History per-lane cache holds two entries, so a many-lane read re-decompresses every window #34, where a two-entry per-lane cache
    (src/store/archive.ts:316, src/store/history.ts:405) makes a many-lane read
    re-decompress every window. Both are the same surface seen from opposite ends —
    one is read cost, this is write cost.
  • Nothing else in the suite is near the timeout cliff: after this case the next
    slowest are 1.4 s and 1.1 s.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions