fix(#627): support ClickHouse 24.8 meta-less result streams in Table view

[BorisTyshkevich]

What & why

Closes #627.

ClickHouse 24.8 never emits the {"meta":[...]} record for JSONStringsEachRowWithProgress /
JSONEachRowWithProgress (upstream added it in ClickHouse PR #74181, merged 2025-01-06,
postdating 24.8). applyStreamLine() learned column names only from meta, so
result.columns stayed empty and every successful row became [] — the UI rendered a
silently empty result while the wire response carried the data.

This implements the issue's limited, data-safe Level-1 support contract: when a row
arrives while result.columns is empty, establish name-only columns from
Object.keys(row) with the explicit unknown-type sentinel type: '', then store every
row through that established order. No value-based ClickHouse type inference, no
auxiliary metadata query, no version branch, no Column.type shape change.

Support contract: query execution and faithful Table result display are supported on
24.8. Features requiring ClickHouse result-type metadata are not guaranteed there.

Production footprint is one file (src/core/stream.ts). The audit below found that
no other production change was needed: every result-type consumer already fails closed
on ''.

Layer ownership

The fallback is SQL Browser result policy, so it lives in src/core/stream.ts, keeping
the settled #630 Phase 3 split intact — packages/clickhouse-http's streamLines() still
only decodes what the server actually sent and never synthesizes metadata. Nothing under
packages/clickhouse-http/**, src/net/**, src/application/**, or src/ui/** changed.

Contract coverage (all 9 acceptance criteria, none deferred)

#	Criterion	Proof
1	Meta-less rows populate columns, preserve values	stream.ts fallback + true-EOF test + live evidence
2	Unknown types are exactly type: '', no inference	literal sentinel + synthetic-looking-value test (big int, decimal, datetime, UUID, bool, enum-like — all stay '')
3	Multiple rows through EOF render correctly	stable-order test + accumulator→renderGrid fidelity test + live two-row proof
4	Meta-first typed behavior unchanged	meta arm untouched; test asserts metadata order stays authoritative when it deliberately differs from row-key order
5	Type-dependent consumers don't crash/discard/fabricate	audit + regressions in results, chart-data, kpi, logs, variable-options, panel-cfg, spec-editor
6	Row-cap / progress / exception unchanged	existing suite + meta-less cap and mixed-stream cases
7	Live verification on both pinned 24.8 images	docs/evidence/627/ (see below)
8	24.8 documented as limited, not full support	new ## ClickHouse server compatibility README section; #71 body records the decision (and stays open)
9	ADR-0005 / #585 cross-reference this as the production fix	ADR follow-up note; ADR remains Rejected

Result-type consumer audit

isNumericType('') → false; parseClickHouseType('') → null; chartStripType('') → ''
(so chartRole never returns numeric/time and autoChart() yields null); logs type
regexes match nothing; isOptionColumnType('') → false (unknown metadata is not treated
as String); the cell-detail type badge is truthiness-gated; autoPanel falls through to
Table. param-type.ts/isSupportedOptionScalar and the schema/catalog c.type sites are
authored-declaration and catalog consumers, not result columns, and are excluded.

One nuance worth naming: resolveLogsShape resolves an explicit cfg.time/cfg.msg by
column name, never by type — unlike convention-based auto-detection, which fails closed on
''. So an authored Logs panel now resolves against a meta-less result (formatting an
untyped string as its time column). That is that consumer's pre-existing unknown-type
behavior — it neither throws, discards row data, nor fabricates a type — so #627's
degraded-functionality contract permits it. It is now pinned by a test rather than left
undocumented.

Invariant verification

Every invariant was sabotage-checked — the mutation was applied, the expected failure
confirmed, and the original bytes written back (never git checkout --):

Invariant	Sabotage	Result
A meta-less row is never discarded for lack of metadata	remove the fallback	EOF test + 3 others fail with empty rows
Unknown metadata never becomes a fabricated type	type: '' → 'String'	5 stream tests + grid fidelity test fail
The first row fixes column order	rebuild columns every row	reversed-key-order + meta-first tests fail
Meta-first keeps server types	blank/reorder metadata types	strengthened meta-first test fails
Precision strings survive without coercion	wrap values in Number(...)	4 stream tests + grid precision test fail (NaN)
Cap/progress/exception unchanged	bypass the row cap	meta-first + meta-less cap tests fail
Consumers fail closed on ''	force isNumericType('') true	chart no-auto-chart + grid no-.num assertions fail
Every stored row's width equals columns.length	revert the zero-key guard	new zero-key regression fails ([[], ['srv-7','ok']])

The last one came out of internal review (below), which is why it landed as a separate commit
rather than in the original implementation.

Live ClickHouse 24.8 verification

Both pinned digests from the #585 compatibility matrix, run against the real production
decoder — streamLines() → newResult()/applyStreamLine() — over the actual captured
bytes, asserted against independently declared literals:

Image	version()	meta in stream	Columns/rows
clickhouse/clickhouse-server@sha256:1ffa82ed…	24.8.14.39	none	exact match, all type: ''
altinity/clickhouse-server@sha256:d0c45645…	24.8.14.10547.altinitystable	none	exact match, all type: ''

Raw NDJSON and normalized output are committed under docs/evidence/627/ with the full
procedure. The verification is deliberately one-off with committed evidence, not a new
permanent Docker-matrix harness: these are two immutable historical digests, and the #585
spike harness was itself retired in #630 Phase 8.

The precision corpus is the point of the query — 9007199254740993.12345678901234567890
(beyond IEEE-754 exactness), -9007199254740993.00000000000000000001, and the
lexically-significant 001.2300 / 0002 all survive byte-for-byte to rendered Table cells.

Known, deliberately unfixed edge (named, not overlooked)

A row-then-late-meta stream would leave already-stored rows bound to row-key order
while headers flip to meta order, so a differing order could misalign row 0. This is
not reachable in either …WithProgress format (meta, when present, is always the
first line), the json.meta arm is untouched by this PR, and guarding it would require the
extra StreamResult state the approved plan explicitly forbids. Before this PR the same
sequence produced an empty row 0 instead; either way no values are discarded. Recorded
here rather than silently left as an unexamined assumption.

Internal review

One targeted read-only pass (Medium risk) produced 3 real findings, all fixed in edcb8ba:
the unsound zero-key establishment sentinel (treated as a missing invariant, not a spot
patch); the resolveLogsShape gap in the audit's exhaustiveness claim (test-only — the
conclusion held); and three ADR-0005 passages still asserting the defect in the present
tense (shifted to past tense with a pointer to the follow-up note — historical observations
and docs/evidence/585/** are untouched, and the ADR remains Rejected).

Checklist

• npm test passes (per-file coverage gate 100/95/90/100 held; src/core/stream.ts at 100/100/100/100)
• Tests added/updated in the same change as the code
• npm run build succeeds (single-file dist/sql.html)
• Layers kept honest — pure result policy in src/core/, package protocol mechanics untouched
• No new runtime dependency
• README + CHANGELOG.md ([Unreleased] → Fixed) updated
• Reconciled affected tracked work: Document the supported-browser matrix (browsers, ClickHouse versions, IdP requirements) #71 body (24.8 = limited support, stays open), ADR-0005 follow-up, docs/evidence/585/README.md (also drops its instruction to run the retired check:client-spike:evidence script), docs/ARCHITECTURE.md, .wiki/Architecture.md, .wiki/Decisions-and-Roadmap.md. Roadmap Roadmap to 1.0.0 #68 needs no change — this fulfils a decision inside existing Document the supported-browser matrix (browsers, ClickHouse versions, IdP requirements) #71 work rather than reshaping a track.

Test/gate results

Full explicit gate green (.npmrc sets ignore-scripts=true, so the chain is run
explicitly rather than relying on pretest): check:types, check:arch (1899 files, 10
rules, no violations), check:schemas, check:examples, 7550 unit tests, build (counts
are for the final head 97d65a0; three review-pass commits landed after this body was first
written, adding 2 tests).

E2E: Chromium fully green; WebKit showed one page.goto 30 s timeout, but a different
test on each of two full runs (dashboard-tree.spec.js:1124, then variable-tab.spec.js:127),
and the affected spec passes 12/12 in isolation. Both are features this diff does not
touch, and it adds no App-shape, fixture, or import-graph change — the known WebKit
under-parallelism flake, not a regression.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LwFPT465eDJqYcRa8HGNLz

---

Review record

Plan: ChatGPT-authored, approved by Fable/high on pass 3 of 5.
Code review: 3 formal passes in one ChatGPT conversation, certified VERDICT: SHIP at
97d65a0. Pass 1 → 1 accepted finding (stale evidence provenance, fixed e94b7c5, which also
added the permanent replay test). Pass 2 → 2 accepted findings, both caused by pass 1's own fix
(a SHA pinned as "final head"; a "byte-identical" overstatement that is really structural
equality) — fixed e9ca678. 97d65a0 then removed review chronology from the permanent test
file and evidence note on the coordinator's own initiative, since those two findings shared one
root cause: this evidence doc asserting things about code state it cannot keep true. Pass 3 → no
accepted findings; one non-blocking bookkeeping observation about this body's own commit/test
counts, corrected above.

[BorisTyshkevich]

ChatGPT review pass 1

Reviewed head: edcb8ba7cdeed3f57427c01ab241d98dcc1c22fa

Findings

1. [P2] Reconcile the committed live-evidence provenance with the final production head.

   docs/evidence/627/README.md says the tested commit is 036760afe565db89a43f112fa37db3bf7f192257 on wip/627-metaless-stream-columns and that src/core/stream.ts's meta-less fallback "is unchanged since." That is no longer true at the reviewed head: edcb8ba7cdeed3f57427c01ab241d98dcc1c22fa subsequently changed the production json.row path by adding the zero-key-row guard. The guard looks sound and does not affect the committed two-nonempty-row corpus, but the evidence file now makes a false exact-provenance claim, and acceptance criterion 7 is explicitly evidence-backed.

   Please make the evidence record precise before merge: preserve the historical raw captures, but distinguish the original live-capture SHA from current-head validation; correct the stale branch name; and record validation against the exact final production SHA for both committed captures (replaying both raw NDJSON files through the current head's real streamLines() → applyStreamLine() path is sufficient to prove the code-side delta, or rerun both pinned images). The coordinator's current-head OSS re-run is useful evidence, but it is not reflected in the committed record, and the Altinity row likewise has no current-head provenance there.

Other adversarial checks

• The zero-key guard itself is defensible: it fires only before columns are established, discards no field values, and current ClickHouse rejects an empty SELECT projection; it also preserves the stored-row-width invariant.
• The known late-meta sequence is not a blocker for these formats: current ClickHouse emits meta from the format prefix before rows when metadata exists.
• I found no persistence leak that turns type: '' into authoritative saved metadata: lastSuccessfulResultColumns is an in-memory tab snapshot used by Spec completion, while saved queries/workspaces do not serialize it.
• The explicit Logs-config behavior is acceptable under the degraded-functionality contract: it resolves authored roles by column name and does not fabricate a ClickHouse type; auto-detection still fails closed on ''.
• The new grid fidelity assertions use independently declared literals rather than deriving the expected values from result.rows.

I could not run focused tests locally because this review runtime could not resolve github.com for git clone; I therefore reviewed the canonical 22-file PR patch and surrounding sources through GitHub. The exact reviewed head has eight successful CI jobs, including unit/coverage/build and e2e.

VERDICT: REVISE

[BorisTyshkevich]

ChatGPT review pass 2

Previously reviewed head: edcb8ba7cdeed3f57427c01ab241d98dcc1c22fa

Reviewed head: e94b7c506a0ccd6c4e6b339a048b4265206cd6e6

Reassessment of pass-1 finding

The substantive part is fixed. Commit e94b7c5 adds tests/unit/evidence-627-replay.test.ts, which replays both committed raw 24.8 captures through the current checkout's real streamLines() -> applyStreamLine() path. The new-head CI run executes that spec (2/2) inside the 7,550-test suite, so the OSS and Altinity captures now have current-code replay coverage. src/core/stream.ts itself is unchanged from edcb8ba7.

Findings

1. [P2] The evidence README still labels edcb8ba7 as the branch's final head, so the provenance text is already stale again.

   docs/evidence/627/README.md now correctly distinguishes the live-capture SHA 036760a from later code validation, but it says several times that the branch's final head is edcb8ba7 and that decoder/accumulator behavior is attested at that final head. The canonical PR head I reviewed is e94b7c506a0ccd6c4e6b339a048b4265206cd6e6; e94b7c5 is itself the commit that added this README rewrite and the replay test. Thus the fix repeats the same class of exact-head provenance drift from pass 1, even though the replay test makes the underlying proof sound.

   Please make this future-proof rather than hard-coding another "final head": describe edcb8ba7 as the last production-changing commit / zero-key-guard commit, and describe the replay test as validating the checkout/HEAD under test. If you want a concrete validation record, record that the replay passed at e94b7c506a0ccd6c4e6b339a048b4265206cd6e6 (or its CI run) without calling that SHA the permanent final head.

2. [P3] "Byte-identical" overstates what the replay test asserts.

   The new spec JSON.parses each committed *.normalized.json file and compares metaSeen, columns, rows, error, and capped semantically. Those are all fields in the current normalized artifact, so I do not see a data-fidelity hole here, but it is not a byte-for-byte comparison of the normalized JSON file. The README and test comments repeatedly say "byte-identical". Change that wording to "structurally/exactly equal normalized result" (or actually serialize/canonicalize and compare bytes if byte identity is intended).

Complete-PR regression reassessment

• The old production behavior remains as reviewed: first non-empty meta-less row establishes Object.keys(row) order with type: ''; subsequent rows map through that order; meta-first behavior remains authoritative.
• The zero-key guard remains sound for the stated invariant and is unchanged in this pass.
• I still found no persistence/share/export path that promotes type: '' into authoritative saved ClickHouse metadata.
• Explicit Logs cfg behavior remains name-based and permissible under the degraded contract; auto-detection still fails closed on unknown types.
• ADR-0005 remains Rejected and the historical ADR-0005: adopt @clickhouse/client-web behind the SQL Browser transport adapter #585 evidence is not rewritten.
• The current PR body is slightly stale operationally (it still mentions a fourth commit / 7,548 tests), while the canonical PR now has 5 commits and CI reports 7,550 tests; this is non-blocking metadata cleanup compared with the evidence-file issue above.

I could not execute the focused tests locally because this runtime still cannot resolve github.com (git ls-remote fails at DNS). I therefore re-reviewed the canonical 23-file PR and the exact edcb8ba7... -> e94b7c5... delta through GitHub, and verified the new-head CI run is green, including the new evidence replay test.

VERDICT: REVISE

[BorisTyshkevich]

ChatGPT review pass 3

Previously reviewed head: e94b7c506a0ccd6c4e6b339a048b4265206cd6e6

Reviewed head: 97d65a004cf84047071725c35749911e7eaddc1f

Reassessment of earlier findings

All earlier findings are resolved.

• Pass 1 provenance finding: resolved. The committed live-capture SHA remains historical provenance, while tests/unit/evidence-627-replay.test.ts binds both committed OSS and Altinity raw captures to the code actually under test by replaying them through the real streamLines() -> applyStreamLine() path.
• Pass 2 [P2] stale “final head” wording: resolved in e9ca6788bb46e5f28c0e0060db856c44cc0cb7b2. edcb8ba7 is now identified by the stable role “zero-key-guard commit”, and current-code attestation is explicitly tied to whichever checkout/HEAD runs the replay test rather than a SHA claimed to be permanently final.
• Pass 2 [P3] “byte-identical” overstatement: resolved in the same commit. The evidence README and replay-test header now accurately describe deep/structural equality of the parsed normalized result rather than byte-for-byte equality of the JSON file.

The follow-up 97d65a004cf84047071725c35749911e7eaddc1f chronology cleanup does not overreach. It changes only stable explanatory prose/comments and the describe name in docs/evidence/627/README.md / tests/unit/evidence-627-replay.test.ts; it preserves the useful technical fact that edcb8ba7 introduced the zero-key guard, and it changes no assertion, production code, raw NDJSON, or normalized evidence.

Complete-PR regression reassessment

I re-reviewed the complete 23-file PR. The production footprint is still only src/core/stream.ts, whose behavior is unchanged from the previously reviewed head: the first non-empty meta-less row establishes Object.keys(row) order with type: ''; later rows use that established order; the zero-key guard preserves the stored-row-width invariant; and meta-first behavior remains authoritative. I found no regression in data fidelity, row-cap/progress/exception handling, type-sentinel consumers, persistence/share/export behavior, explicit Logs handling, or Table rendering.

The support/documentation side is still internally consistent: ClickHouse 24.8 is documented as limited support, #71 records that decision while remaining open, ADR-0005 remains Rejected, and the #585 evidence remains historical rather than being rewritten.

Current-head CI is green: test, e2e, bundle, Docker smoke, size, and aggregate gate completed successfully. The test job ran 221 files / 7,550 tests, including the evidence replay (2/2), and src/core/stream.ts remains at 100/100/100/100 coverage. I could not rerun focused tests in this sandbox because direct GitHub DNS resolution is still unavailable, so I used the canonical PR/head and its current CI execution as the executable verification source.

No actionable findings remain. One optional bookkeeping cleanup is that the PR description still says the zero-key change made this the “fourth commit” and still reports 7,548 tests; the canonical PR now has 7 commits and current CI reports 7,550 tests. That metadata drift does not affect implementation or merge readiness.

VERDICT: SHIP