Repository navigation
Preview more SQL in Instant SQL and add sampled previews - #15
Merged
Merged
Conversation
The read policy now verifies CTE bodies, cast target types, window functions, view definitions (recursively, as DuckDB binds them) and qualified tables and columns, resolving every name through a catalog retriever configured like the binder's. Refusals state their reason. Adds the policy contract to the runtime smoke test and documents it.
These operators parse to function names that bind through the search path, so they go through the same built-in check as named functions and a macro that shadows one is refused.
…tions A DuckDB database attached over HTTP or S3 still reports the catalog type duckdb, so the policy now also requires the database to be in memory or at a local path. Reading a remote database is network I/O, which a preview would repeat every time typing pauses.
One scalar repeat call can build a value of any size within the notebook's preview time budget, before the result byte cap applies. The repeat table function stays accepted because its rows are bounded. The docs now describe how the host bounds a preview's rows, bytes and execution time.
sqlnb_sample_plan names the one table a sampled preview would sample and the percentage. sqlnb_read_query takes that table and percentage, verifies the statement as before, and puts a seeded Bernoulli sample on the verified reference only while that table is still the statement's sole read of a table of at least 1,000,000 rows.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Widens what Instant SQL can preview and adds an opt-in sampled tier for statements that are too slow to preview exactly.
The read policy now verifies CTE bodies, cast target types, window functions, view definitions (recursively, as DuckDB binds them), qualified tables and columns, and the
LIKE,GLOB, bitwise and list operators, resolving every name the way the binder does so a macro that shadows a built-in is refused. Refusals state their reason. Two cases are newly refused: a DuckDB file attached from a remote location, since a preview would repeat the network read every time typing pauses, and the scalarrepeat, which can build a value of any size before the result byte cap applies.sqlnb_sample_plan(sql)names the one table a sampled preview would sample and the percentage: the largest table of at least 1,000,000 rows that the statement reads once, directly, without a sample clause of its own, with the percentage chosen to keep at least 100,000 rows.sqlnb_read_querytakes that table and percentage, verifies the statement as before, and puts a Bernoulli sample with a fixed seed on the verified reference only while the plan still holds. Bernoulli sampling is per row and repeatable, where DuckDB's default system sampling takes whole vectors and returns nothing from small tables.docs/instant-sql.mddocuments the policy, the sampling rules and how the host bounds a preview's rows, bytes and time. The policy contract is part of the runtime smoke test, which passes on the macOS release build. Linux and Windows have not been built locally.