Skip to content

Preview more SQL in Instant SQL and add sampled previews - #15

Merged
nicosuave merged 5 commits into
mainfrom
instant-sql
Nov 15, 2026
Merged

nicosuave merged 5 commits into
mainfrom
instant-sql

Conversation

@nicosuave

Copy link
Copy Markdown
Member

Widens what Instant SQL can preview and adds an opt-in sampled tier for statements that are too slow to preview exactly.

The read policy now verifies CTE bodies, cast target types, window functions, view definitions (recursively, as DuckDB binds them), qualified tables and columns, and the LIKE, GLOB, bitwise and list operators, resolving every name the way the binder does so a macro that shadows a built-in is refused. Refusals state their reason. Two cases are newly refused: a DuckDB file attached from a remote location, since a preview would repeat the network read every time typing pauses, and the scalar repeat, which can build a value of any size before the result byte cap applies.

sqlnb_sample_plan(sql) names the one table a sampled preview would sample and the percentage: the largest table of at least 1,000,000 rows that the statement reads once, directly, without a sample clause of its own, with the percentage chosen to keep at least 100,000 rows. sqlnb_read_query takes that table and percentage, verifies the statement as before, and puts a Bernoulli sample with a fixed seed on the verified reference only while the plan still holds. Bernoulli sampling is per row and repeatable, where DuckDB's default system sampling takes whole vectors and returns nothing from small tables.

docs/instant-sql.md documents the policy, the sampling rules and how the host bounds a preview's rows, bytes and time. The policy contract is part of the runtime smoke test, which passes on the macOS release build. Linux and Windows have not been built locally.

The read policy now verifies CTE bodies, cast target types, window
functions, view definitions (recursively, as DuckDB binds them) and
qualified tables and columns, resolving every name through a catalog
retriever configured like the binder's. Refusals state their reason.
Adds the policy contract to the runtime smoke test and documents it.
These operators parse to function names that bind through the search path,
so they go through the same built-in check as named functions and a macro
that shadows one is refused.
…tions

A DuckDB database attached over HTTP or S3 still reports the catalog type
duckdb, so the policy now also requires the database to be in memory or at a
local path. Reading a remote database is network I/O, which a preview would
repeat every time typing pauses.
One scalar repeat call can build a value of any size within the notebook's
preview time budget, before the result byte cap applies. The repeat table
function stays accepted because its rows are bounded. The docs now describe
how the host bounds a preview's rows, bytes and execution time.
sqlnb_sample_plan names the one table a sampled preview would sample and
the percentage. sqlnb_read_query takes that table and percentage, verifies
the statement as before, and puts a seeded Bernoulli sample on the verified
reference only while that table is still the statement's sole read of a
table of at least 1,000,000 rows.
@nicosuave
nicosuave merged commit 3434de6 into main Nov 15, 2026
8 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant