fix(grid-core): stop an evicted selection painting rows nobody selected

.changeset/eviction-population-and-gate.md

@@ -0,0 +1,55 @@
+---
+"@pretable/core": minor
+"@pretable/react": minor
+---
+
+Eviction: two publicly-reachable correctness fixes, both of which contradicted
+what the docs promised.
+
+**An evicted selection no longer paints rows the reader never selected.**
+`datasetKey` identifies the QUERY, not the population — deliberately, and
+[the docs](/docs/server-data/lifecycle#datasetkey) tell consumers to keep it
+stable while they page within one result. So an insert or a delete made by
+someone else, upstream of a selection whose own rows are unloaded, arrived with
+the key unchanged and silently re-filled the remembered dataset positions with
+different rows. Reproduced through `<PretableSurface>` with the honesty gate
+fully passing: a `row-1..row-8` selection, both endpoints evicted, five rows
+prepended to the same result, and the returning window painted five rows
+selected — four of which had not existed when the user selected — while the
+eight they did choose painted nothing.
+
+A span now records the population's SIZE alongside its key
+(`PretableIndexedDatasetRowSpan.datasetTotal`, from the exact
+`resultMeta.total.count` the gate already requires), and a mismatch fails closed
+exactly as a key mismatch does: nothing paints from the span, and
+`getCellSelectionSummary()` reports `verified: false` until a window covering
+both endpoints re-measures it. A proven deletion is the one allowance — a total
+short by exactly the rows the engine watched vanish is accounted for, so
+endpoint narrowing still works. What this does **not** catch is a change that
+leaves the size identical; `eviction.mdx` now says so rather than promising
+otherwise.
+
+`ɵPretableIndexedSelectionWindow.datasetTotal` is required, not optional: the
+gate that builds a window cannot pass without an exact total, so an optional
+field would only be a way to fail open by omission.
+
+**One closed-gate revision no longer destroys the selection and the cursor.**
+With `resultMeta.total` reporting `{kind: "estimate"}` for a single render — an
+in-flight count query, a backend that stops counting past 10k — or one revision
+of `processing.sort: "engine"`, a window slide dropped every range and emptied
+the cursor irrecoverably. Restoring the exact total brought neither back.
+Uncontrolled consumers only; a controlled one was accidentally immune because
+the `state.selection` echo re-supplied what the engine had discarded.
+
+A null window was two different situations sharing one representation. The
+engine is now told which: `windowed` says whether the consumer publishes
+`resultMeta.window` at all, independent of any gate, so a windowed grid with no
+window this revision means "I cannot verify", not "those rows were deleted" —
+and it holds the selection and the cursor byte-for-byte instead of asserting a
+deletion it could not have observed. Local mode, where an absent row genuinely
+is a deleted row, is unchanged in every branch and pinned by tests that run the
+same fixture both ways.
+
+`CreateGridUiCoreOptions.getSelectionWindow` is replaced by `getWindowing`,
+which returns both facts from one read so they cannot describe different
+instants.

apps/bench/tests/eviction.spec.ts

@@ -27,6 +27,8 @@ import { expect, test, type Page } from "@playwright/test";
 const ROW_HEIGHT = 48;
 const PAGE_SIZE = 50;
 const WINDOW_START = 5_000;
+/** `TOTAL_ROWS` in `windowed-harness.tsx`, which this drives. */
+const HARNESS_TOTAL_ROWS = 10_000;
 
 /** The selection under test: 11 rows, at dataset positions 5,010–5,020. */
 const SELECT_FROM = 5_010;
@@ -598,6 +600,10 @@ test.describe("a cell selection survives its rows being evicted", () => {
         start: SELECT_FROM,
         end: SELECT_TO,
         datasetKey: "windowed-harness",
+        // The population the positions were measured in. `datasetKey` says
+        // which QUERY; this says how big its result was, which is what
+        // catches somebody else inserting rows above an evicted selection.
+        datasetTotal: HARNESS_TOTAL_ROWS,
       });
 
     // 1a. The evicted endpoint really is gone from the DOM — otherwise

apps/website/content/docs/server-data/eviction.mdx

@@ -32,6 +32,8 @@ Two things it deliberately does not do. It will not name the selected records wi
 
 **The cursor survives too.** While the focused row is unloaded its cell element is gone, so focus parks on the grid's scroll viewport — never on `<body>`, which would drop the reader out of the grid entirely and take the keyboard with them. When the row returns the cursor is back on the same cell, exactly one cell holds it, and an arrow key moves on from the row it was left on rather than from wherever the viewport happens to be parked.
 
+**A revision the grid cannot vouch for holds, rather than discards.** The honesty gate below can shut for a render without anything being wrong: a count query still in flight reports `total.kind: "estimate"`, a backend that stops counting past ten thousand never reports anything else, and a single revision of engine-side sort takes positional authority away. On such a render the grid does not know where the loaded rows sit, and it treats that as not knowing rather than as evidence — the selection and the cursor are held exactly as they were, and the next render that can place them carries on. A grid with no `resultMeta.window` at all is a different case and behaves as it always has: there the whole result arrives on every render, so a row that is absent has genuinely been removed, and the selection prunes.
+
 ## What it costs you
 
 One thing, and it is not optional: **`resultMeta.datasetKey`**.
@@ -40,6 +42,8 @@ Everything above turns on dataset positions, and a position only means something
 
 Because that loss is invisible and the type says nothing about it, a grid that publishes a trusted `window` and no `datasetKey` says so — one `console.warn`, in production builds too, naming what it refused. [The `datasetKey` section on Loading, staleness, errors](/docs/server-data/lifecycle#datasetkey) is where the rule for choosing one lives: change it when the result set changes, keep it stable while you page within one result, and derive it from something that commits rather than from something that ticks.
 
+The exact total the honesty gate already requires does the other half of the job. A key that is correctly held stable across pages cannot report an insert or a delete made by someone else, and the count is what does: a span measured when the population held 20 rows is refused once it holds 25, because the positions it remembers now name different rows. That costs you nothing extra — [Totals and honesty](/docs/server-data/totals) already requires the exact count for the window to be trusted at all — but it is worth knowing that the number is load-bearing for selection and not only for the scrollbar.
+
 ## What it does not do
 
 **It does not decide memory pressure for you.** There is no block limit, no cache ceiling, and no policy you configure — the grid never releases a row on its own and never asks you to. [`telemetry.windowGap`](/docs/server-data/windowing#knowing-when-to-fetch) prompts the fetch half of the decision and says nothing at all about the release half; how many blocks are worth holding is a judgement about your data and your users' machines, and it stays with you.
@@ -48,6 +52,8 @@ Because that loss is invisible and the type says nothing about it, a grid that p
 
 **It does not survive a query change, and must not.** A new `datasetKey` is a statement that the positions now hold different rows, so the spans measured under the old key are refused rather than repainted onto whatever occupies those positions today. A range whose endpoints are both present in the new population is re-measured there and kept — it is fully locatable, so there is nothing to guess — and a range that is only half locatable collapses onto the endpoint that survived, with its span dropped so the smaller count that results is not presented as a proven one. Selection surviving a re-sort is out of scope by design, not by accident.
 
+**It does not notice a change that leaves the population the same size.** Rows inserted or removed upstream of an evicted selection move every position after them while `datasetKey` stays correctly unchanged, so the grid compares the exact total as well: when the size differs from the size a span was measured at, the span is refused, nothing paints from it, and `getCellSelectionSummary()` reports `verified: false` until a window covering both endpoints re-measures it. A revision that removes one row and adds another leaves that count identical, and there the grid still answers from the remembered positions — which by then name the neighbours of the rows the reader chose. It reports `verified: false` throughout, which is the whole of what it can honestly say; distinguishing a re-shuffled population from a scrolled one would take a token that changes on every write, and only your backend can mint that. If your result set is written to while readers hold selections over it, treat `verified: false` as the signal it is and re-derive from a fresh read rather than from the span.
+
 ## See also
 
 - [Windowing](/docs/server-data/windowing) — `resultMeta.window`, the `windowGap` signal that prompts the fetch, and the honesty gate all of this rides.

apps/website/content/docs/server-data/lifecycle.mdx

@@ -92,7 +92,9 @@ Two things worth knowing about that callback. Returning `null` does not suppress
 
 Change it when the result set changes — a new filter, a different sort, a different search — so that interaction state cannot silently reattach itself to a different dataset. Keep it stable while you page within one result: a second page of the same query is more of the same population, not a new one.
 
-The reason is that some interaction state is recorded against positions in the result rather than against row ids, and positions are only meaningful within one population. A re-sort re-fills position 40 with a different row; a filter change re-fills every position. The dataset key is the evidence the engine uses to tell "the reader scrolled" from "this is a different table now", and it fails closed: when the key a selection was measured under and the key the grid now reports disagree — or when there is no key at all — the engine refuses what it recorded rather than repainting it onto whatever rows occupy those positions today. What that costs is a selection that shrinks to the rows actually loaded, visibly. What it buys is never painting a row the reader did not select.
+The reason is that some interaction state is recorded against positions in the result rather than against row ids, and positions are only meaningful within one population. A re-sort re-fills position 40 with a different row; a filter change re-fills every position. The dataset key is the evidence the engine uses to tell "the reader scrolled" from "this is a different table now", and it fails closed: when the key a selection was measured under and the key the grid now reports disagree — or when there is no key at all — the engine refuses what it recorded rather than repainting it onto whatever rows occupy those positions today. What that costs is a selection that shrinks to the rows actually loaded, visibly.
+
+The key answers "is this the same query?" — it does not answer "is this the same set of rows?", and keeping it stable while you page is exactly what stops it from trying. Rows inserted or removed by someone else, upstream of a selection whose own rows are unloaded, arrive with the key unchanged and shift every position after them. The engine compares `resultMeta.total.count` alongside the key for that, and refuses the recorded positions when the population has changed size — so the two together are what keep a returning window from painting rows the reader never selected. Neither can see a change that leaves the size alone; [Eviction](/docs/server-data/eviction#what-it-does-not-do) says what remains.
 
 Deriving it from the query is the usual answer, and the derivation only has to be stable: `JSON.stringify(query)` is a fine key. What it must not be is early or arbitrary. The example above sets its key when a search **commits**, not on each keystroke, because until the new rows arrive the ones on screen still answer the previous search — and a key recomputed per render from something like a timestamp says "different table" on every frame.
 

packages/core/core.api.md

@@ -73,7 +73,7 @@ export interface CreateGridUiCoreOptions<TRow extends object, TRowId extends Pre
     // (undocumented)
     readonly columns: readonly PretableGridUiColumn<TColumnId>[];
     // @internal
-    readonly getSelectionWindow?: () => ɵPretableIndexedSelectionWindow | null;
+    readonly getWindowing?: () => ɵPretableIndexedWindowing | null;
     // (undocumented)
     readonly rowModel: PretableRowModel<TRow, TRowId, TColumns>;
     // (undocumented)
@@ -848,6 +848,7 @@ export interface PretableIndexedCellSelectionSummary {
 // @public
 export interface PretableIndexedDatasetRowSpan {
     readonly datasetKey?: string;
+    readonly datasetTotal?: number;
     readonly end: number;
     readonly start: number;
 }
@@ -1325,12 +1326,21 @@ export type RowOf<TModel> = TModel extends {
 // @internal
 export interface ɵPretableIndexedSelectionWindow {
     readonly datasetKey?: string;
+    readonly datasetTotal: number;
     // (undocumented)
     readonly length: number;
     // (undocumented)
     readonly start: number;
 }
 
+// Warning: (ae-internal-missing-underscore) The name "ɵPretableIndexedWindowing" should be prefixed with an underscore because the declaration is marked as @internal
+//
+// @internal
+export interface ɵPretableIndexedWindowing {
+    // (undocumented)
+    readonly window: ɵPretableIndexedSelectionWindow | null;
+}
+
 // Warning: (ae-internal-missing-underscore) The name "ɵsetLocalRowModelFilterAuthority" should be prefixed with an underscore because the declaration is marked as @internal
 //
 // @internal

packages/core/src/public_api.ts

@@ -145,6 +145,9 @@ export type {
 // tells an evicted row from a deleted one, and `@pretable/react` is the only
 // caller that knows whether the honesty gate has passed.
 export type { PretableIndexedSelectionWindow as ɵPretableIndexedSelectionWindow } from "@pretable-internal/grid-core";
+// Named by `CreateGridUiCoreOptions.getWindowing`'s signature, so it ships
+// alongside the window type it wraps.
+export type { PretableIndexedWindowing as ɵPretableIndexedWindowing } from "@pretable-internal/grid-core";
 // Re-declares who selected the loaded records on a model this package created.
 // Exported because `processing` is a render-time prop on `@pretable/react`, so
 // the authority a rows-mode model is built with can change while it is alive,

packages/grid-core/src/__tests__/grid-ui-core.test.ts

@@ -719,7 +719,7 @@ describe("UI-only grid core", () => {
   /**
    * A grid whose consumer serves a moving WINDOW over `all`, exactly as the
    * windowed-data contract describes: `setRows` gets the loaded slice and
-   * `getSelectionWindow` reports where that slice sits in the dataset.
+   * `getWindowing` reports where that slice sits in the dataset.
    *
    * Every eviction test below drives gestures through the real store rather
    * than calling `reconcileIndexedSelection` with a hand-built fixture. That
@@ -737,12 +737,15 @@ describe("UI-only grid core", () => {
       readonly start: number;
       readonly length: number;
       readonly datasetKey?: string;
+      readonly datasetTotal: number;
     } | null = null;
     const rowModel = createLocalRowModel({ rows: [], columns: modelColumns });
     const grid = createGridUiCore({
       rowModel,
       columns: visualColumns,
-      getSelectionWindow: () => selectionWindow,
+      // Windowed throughout: `windowing` is non-null even before the first
+      // slide, when the window itself is still unknown.
+      getWindowing: () => ({ window: selectionWindow }),
     });
     // A published `datasetKey` by default: spans are fail-closed on it (see
     // `spanReadableInWindow`), so a windowed consumer that never sets one
@@ -754,7 +757,10 @@ describe("UI-only grid core", () => {
       datasetKey = "population-1",
     ) => {
       rowModel.setRows(all.slice(start, start + length));
-      selectionWindow = { start, length, datasetKey };
+      // `total`, not `length`: the population is the whole of `all`, and the
+      // window is a slice of it. Publishing the slice's own size here would
+      // make every slide look like a population change.
+      selectionWindow = { start, length, datasetKey, datasetTotal: total };
       grid.observeRowModelRevision(rowModel.getState().snapshot.revision);
     };
     /** The shape every surface gesture builds: a fresh range, ids only. */
@@ -953,10 +959,16 @@ describe("UI-only grid core", () => {
     const grid = createGridUiCore({
       rowModel,
       columns: visualColumns,
-      getSelectionWindow: () => ({
-        start: 0,
-        length,
-        datasetKey: "population-1",
+      getWindowing: () => ({
+        window: {
+          start: 0,
+          length,
+          datasetKey: "population-1",
+          // The whole dataset is resident here, so the population size and
+          // the window length are the same number -- and a deletion moves
+          // both, which is what makes the narrowing below provable.
+          datasetTotal: length,
+        },
       }),
     });
     grid.observeRowModelRevision(rowModel.getState().snapshot.revision);
@@ -1023,6 +1035,7 @@ describe("UI-only grid core", () => {
             start: 10,
             end: 40,
             datasetKey: "population-1",
+            datasetTotal: 200,
           },
         },
       ],