Skip to content

server: the live stream's cursor describes what it read, not the file - #120

Merged
Shashankss1205 merged 1 commit into
mainfrom
fix/live-stream-size-cursor
Sep 26, 2026
Merged

Shashankss1205 merged 1 commit into
mainfrom
fix/live-stream-size-cursor

Conversation

@Shashankss1205

Copy link
Copy Markdown
Collaborator

The build_snapshot read-then-stat race from the 2026-08 sweep. It was recorded as suspected and never demonstrated — so the first thing here is the demonstration.

The cursor

frames() polls the trace file's size, rebuilds a snapshot when it changes, and stores the snapshot's own size as "everything up to here has been sent":

last_size = snapshot.size if snapshot.size else size

build_snapshot took that number from path.stat() after reading the events. Those disagree in the direction that matters, two ways:

  1. TailRecorder cuts at the last newline, so a half-written final line is excluded from the events but counted by st_size.
  2. The run is appending while the snapshot is built — that is the premise of a live view — so bytes can land between the read and the stat.

Only (2) loses whole events, since the bytes (1) omits cannot parse into one yet.

Why that wedges rather than lagging

A cursor past what was read means the next poll sees an unchanged file size and rebuilds nothing. If the writer has stopped, the file never changes size again, so the stream never rebuilds and never learns the run ended.

Reproduced against the old code with a single append landing in the window:

  cursor stored : 375
  final size    : 375
  done          : False
  WEDGED (cursor == final and not done): True

That pair is the bug: frames() has no reason to rebuild and nothing left to learn from. The page shows a finished run as running for as long as it stays open.

The fix

TailRecorder.read_tail() returns the events and the byte count they came from. read_events() keeps its old signature over it, since that is what every other caller uses. build_snapshot uses the count for size and reads only the mtime from the file's current state — which is safe to be too new: it makes a run look more recently active, delaying an idle verdict by one poll rather than stopping the stream.

Because the cursor now falls short of the file, the next poll rebuilds and the missed event arrives. That is the difference between a one-poll delay and a stream that never recovers.

Tests

Nine, including the two that guard against over-correcting:

  • test_a_quiet_complete_trace_reports_the_whole_file — the ordinary case, where the cursor is the file size and the stream must still settle.
  • test_an_empty_or_missing_trace_is_still_a_waiting_snapshot — the URL is posted before the run starts writing.
  • test_a_finished_run_stops_reporting_itself_as_running — the end-to-end symptom, asserting the wedged pair cannot occur and that the follow-up build reports done.

One correction worth recording: my first version of that last test asserted done on a bare stop phase, which is not what compose_snapshot reads — it needs a termination_reason in the state delta. The test now appends a genuine terminal event.

Verification

194 tests across the live, server, trace and replay files pass; ruff check clean; figure refreshed to 2,189 with GRAPHARC_UPDATE_FIGURES (#119's first real use). The full matrix is left to CI.

🤖 Generated with Claude Code

`frames()` polls the trace's size, rebuilds a snapshot when it changes, and
stores that snapshot's `size` as "everything up to here has been sent":

    last_size = snapshot.size if snapshot.size else size

`build_snapshot` took that number from `path.stat()` *after* reading the
events, and the two can disagree in the direction that matters. The read
stops at the last newline, and -- this being a live view -- the run is
appending while it happens, so bytes can land between the read and the stat.
The cursor then claims events the snapshot never saw, the next poll sees an
unchanged file size and rebuilds nothing, and if the writer has stopped the
file never changes size again. A finished run streams as a running one for
as long as the page is open.

Demonstrated against the old code rather than argued: with one append landing
in that window, the cursor came back equal to the file's final size while
`done` was still False -- the exact pair that leaves `frames()` with no
reason to rebuild and nothing left to learn from.

`TailRecorder.read_tail()` now returns the events *and* the byte count they
came from; `read_events()` keeps its signature over it, since that is what
every other caller uses. `build_snapshot` uses the count and reads only the
mtime from the file's current state, which is safe to be too new: it makes a
run look more recently active, delaying an idle verdict by one poll rather
than stopping the stream.

Nine tests, including the ordinary quiet-and-complete case -- where the
cursor is the file size and the stream must still settle -- and the
half-written final line, which is excluded from the count because bytes after
the last newline cannot parse into an event yet.

This is the `build_snapshot` read-then-stat race from the 2026-08 sweep. It
was recorded as suspected and never demonstrated; it is demonstrated now.

Verified: 194 tests across the live, server, trace and replay files, ruff
clean, figure refreshed to 2,189.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Shashankss1205
Shashankss1205 merged commit 652d979 into main Sep 26, 2026
7 checks passed
@Shashankss1205
Shashankss1205 deleted the fix/live-stream-size-cursor branch September 26, 2026 09:12
@Shashankss1205 Shashankss1205 mentioned this pull request Sep 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant