Skip to content

fix(renderer-dom): price spacer rows at what rows actually measure - #465

Merged
blove merged 4 commits into
mainfrom
blove/spacer-accuracy
Aug 17, 2026
Merged

fix(renderer-dom): price spacer rows at what rows actually measure#465
blove merged 4 commits into
mainfrom
blove/spacer-accuracy

Conversation

@blove

@blove blove commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

The windowed spacer never consulted a retained height, so the scroll extent was systematically short on exactly the grids this feature exists for — and a published doc claimed the opposite.

The defect

const leadingHeight = Math.max(0, (spacers?.leadingRows ?? 0) * defaultRowHeight);

Its own comment said so: "Row counts, not pixel heights." The retained-measurement cache is keyed by row identity; getWindowSpacers supplies only counts. The controller knows how many rows are out there, never which — so the two systems had never met. And since estimate() floors at defaultRowHeight, every evicted row was understated whenever rows wrap.

The gate, verbatim, before any production change

A new test drives createRowLayoutController — never planViewport. 50 loaded rows, leadingRows: 5000, trailingRows: 4950, defaultRowHeight: 30, heights 30 + ((i*7)%23), giving a 40.96px measured mean against a 30px default:

the leading spacer at the measured mean: expected 150000 to be close to 204800, difference 54800
the whole extent at the measured mean:   expected 300548 to be close to 409600, difference 109052
5,050 evicted rows at their measured mean: expected 151500 to be close to 206848, difference 55348
the extent after the window moved:        expected 300155 to be close to 409207, difference 109052

The second test evicts for real: model.setRows moves the window to rows 50–99, the first fifty become retained tombstones through the cooperative replacement builder, and it asserts no loaded row is measured — so the only possible source of calibration is the evicted rows' retained heights.

Why a new test rather than extending the existing one: layout-core/src/__tests__/eviction-anchor.test.ts calls planViewport({ leadingHeight: sumHeights(...) }) — the exact sum, hand-computed by the test. planViewport is pure and uses what it is handed; the controller never computes that number. That test has looked like proof of this property for the whole project and never was.

The fix — a calibrated mean

RowHeightIndex.getMeasuredHeightMean(): number | undefined — the mean across the whole measurement cache, visible and retained/tombstoned, undefined when empty. The controller reads it once per prepareWindow and prices spacer rows at it, falling back to defaultRowHeight. No consumer API change.

How the mean survives copy-on-write and cache eviction: not by threading a running total. Every HAMT node now carries sum beside the count it already carried, derived at construction from the same children. A running total would need correct deltas in measure (an overwrite must subtract the old value), apply, the retention-eviction loop and the replacement builder — five independent chances to drift on a persistent structure. A structural aggregate has none: a node that exists has the right sum, a rebuilt node recomputes it. Non-numeric values weigh nothing, so the visible-key map sums to zero and the tombstone map's ticket sum is never read.

Proven

Mutation Reddens
spacer priced at defaultRowHeight again both new tests with the exact gaps above, plus the anchoring test (expected 150000 to be 410000)
collision node drops its own entry's weight means measurements that share a hash — expected +0 to be 56
hashLeaf sum: 0 means the measurement cache — expected +0 to be 50, and the bounded-oracle replay test

Cold start is exact, not approximate. A control asserts a windowed grid with nothing measured gets leadingHeight === LEADING_ROWS * DEFAULT_ROW_HEIGHT and a matching extent with toBe, not toBeCloseTo. With no spacer every term multiplies by zero, so the unwindowed path is unchanged by construction — confirmed by 127 pre-existing renderer-dom tests and 1,216 react tests.

One existing test changed, and it got stricter

window-spacer-coordinates.test.ts's anchoring test asserted after.scrollTop === before.scrollTop + 40. That measurement is the grid's first, so it is also the first calibration sample: 5,000 spacer rows reprice from 30px to the single 82px height observed, and the spacer grows 260,000px in the same commit. The relation was probed rather than guessed — the row's on-screen position holds at −11 exactly, and the offset is before.scrollTop + 40 + (after.leadingHeight − before.leadingHeight). Both terms are now asserted, plus the spacer's new value.

Known behaviour change — worth a human's eye

A single-sample mean is volatile. During initial mount the extent visibly swings until a viewport's worth of rows is measured. Anchoring holds the focused row in place; the scrollbar thumb does move.

This is inherent to Option A and no test pins it as good or bad. It is still an improvement on the previous state, where the extent was both wrong and changed every time the window moved — this one converges. If it reads badly on a real grid, the mitigation is to withhold calibration until N samples, or blend toward the mean.

Claims corrected

  • eviction.mdx"reproduces the region's height precisely" is gone. The new claim: the grid knows how many rows are out there, never which, so it prices them at the mean of every row it has measured (retained heights included), falling back to the theme's row height. It is an estimate and stays one; geometry does change across an eviction and again as the mean improves; the anchor absorbs it.
  • windowing.mdx — a spacer is a row count, so the extent is accurate about how many rows and approximate about how tall.
  • Spec §4 — the cost model was inverted. It now reads: every eviction costs an anchor correction, sized by the gap between real heights and the mean. It also records why the earlier version was wrong (it assumed the spacer is built from retained heights; it is fed counts) and that a planViewport-level test asserts over a number production never computes.
  • Spec §3 — the unbuilt block-collapse policy is flagged: a retained per-block total has nowhere to go while the spacer is fed counts. Redesign before building.

The plan's "about half" wrap-ratio figure was dropped from the docs — there is no test for a specific factor, and a number without a test is how the previous claim got there.

Verification, against baselines measured on this tree first

baseline after
renderer-dom 127 / 7 files 130 / 8
layout-core 98 100
grid-core 124 124
react 1216 1216
app-bench 155 155
playwright (bench e2e) 28 28
website 554 554

Repo-wide typecheck, lint and prettier clean. pnpm buildpnpm apipnpm api:check in that order, no .api.md diff. Playwright ran on an isolated port, not the shared 4173. All pipelines used set -o pipefail, and the typecheck exit code was re-measured un-piped after a grep-masked ambiguity.

Not fixed here

Exactness needs Option B — getWindowSpacers carrying heights or row keys rather than counts — which is a public API change that pushes bookkeeping onto every consumer. Not taken; now documented as the prerequisite in spec §3 and §4.

🤖 Generated with Claude Code

@vercel

vercel Bot commented Aug 17, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
pretable Ignored Ignored Aug 17, 2026 4:18am

Request Review

@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Vercel preview ready

Preview: https://pretable-glz0895mw-cacheplane.vercel.app
Commit: 0a88d15075b454fbe4285f5bb97b43524275e9d3

Updated automatically by the deploy-preview job.

blove and others added 4 commits August 16, 2026 21:15
The spacer is rowCount x defaultRowHeight and never reads a retained
height, so wrapped grids understate the extent by ~2x. The test that
appeared to prove otherwise feeds planViewport a number the controller
never computes.
A windowed grid's spacer arrives as a row COUNT — `getWindowSpacers` reports
how many rows sit outside the loaded window and never which — so the only
calibration available to whoever draws it is a per-row average. The index is
the one thing that holds real measurements, so the mean belongs here.

`getMeasuredHeightMean()` returns the mean of the whole measurement cache:
rows currently visible and measured, plus the retained heights of rows that
have left the view. Estimates are excluded, because `apply` already drops a
row's cached entry when it re-estimates. `undefined` when nothing has been
measured, so a caller falls back to its own default rather than to a mean of
no samples.

Aggregated STRUCTURALLY. Every hash node now carries `sum` beside the `count`
it already carried, derived at construction from the same children. The
alternative — a running total threaded through `measure`, `apply`, the
retention eviction loop and the cooperative replacement builder — has five
independent chances to go stale on a copy-on-write index, and the aggregate
has none: a node that exists has the right sum, and a rebuilt node recomputes
it. Non-numeric values weigh nothing, so the visible-key map sums to zero;
only the measurement map's sum is ever read.

The 500-step measurement-cache oracle now checks the mean against the same Map
it already checks the cache count against, so re-measures, re-estimates,
tombstone eviction past the bound and full replacement rebuilds are all
covered. Dropping the entry's own weight from a collision node, and zeroing a
leaf's, each redden it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`prepareWindow` turned a spacer's row count into pixels by multiplying by
`defaultRowHeight`. Its own comment said so — "Row counts, not pixel heights."
That is the region's real height only on a grid whose rows are all the default
height; on a grid whose rows wrap it understates the entire scroll extent by
the ratio between a wrapped row and the unwrapped default, which is the case
this feature exists for. The retained-measurement cache knew exactly what those
rows were worth and was never consulted for geometry, because it is keyed by
row identity while the spacer arrives as a count.

The spacer's rows are now priced at `getMeasuredHeightMean()`, falling back to
`defaultRowHeight` until something has been measured — so a cold grid, and
every grid with no window, is byte-for-byte unchanged. Read once from the
incoming root, like `spacers` itself: the estimate passes below add estimates,
which never enter the measurement cache.

Option A of the plan, deliberately: exactness would need `getWindowSpacers` to
carry heights or row keys rather than counts, which is a public API change that
pushes bookkeeping onto every consumer and that a consumer windowing over rows
it has never rendered cannot satisfy. This stays an estimate, and anchoring is
what absorbs the residual.

The new test drives `createRowLayoutController`, not `planViewport`, because
`planViewport` is handed the leading height it would be asked to check —
which is why `eviction-anchor.test.ts` looked like proof of this property for
the whole eviction project and was not. Reverting the calibration alone
reddens it by 54,800px on the leading spacer and 109,052px on the extent.

One existing assertion moved: the first measurement on a windowed grid is also
the first sample the spacer is calibrated from, so the scroll offset now
follows the row's growth AND the spacer's. The row's on-screen position, which
is the claim that test exists for, is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`eviction.mdx` claimed that "where the retained heights are exact the spacer
reproduces the region's height precisely, so the scroll extent is the same
number after the eviction as before it and nothing shifts at all." No code path
could produce that: the spacer is fed row counts, and there is no arrangement
of retained heights that makes a count exact. Replaced with what it computes —
the mean of every measured height, the theme's row height until one exists —
and what that costs: the geometry does change across an eviction, and the
viewport anchor absorbs it.

`windowing.mdx` gains the one sentence a reader of the extent claim needs: a
spacer is a row count, so the extent is accurate about how many rows are out
there and approximate about how tall they are.

The spec's §4 cost model was inverted — it had evicting measured rows free and
only unmeasured rows costing an anchor correction. Every eviction costs one.
§3's block collapse is unbuilt and its premise does not hold either: a retained
per-block total has nowhere to go while the spacer is fed counts, so it is
flagged for redesign rather than left as a plan.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@blove
blove force-pushed the blove/spacer-accuracy branch from 1daff4e to 0a88d15 Compare August 17, 2026 04:18
@blove
blove merged commit 01a7d60 into main Aug 17, 2026
20 checks passed
@blove
blove deleted the blove/spacer-accuracy branch August 17, 2026 04:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant