Skip to content

Keep complete results in an Arrow result cache - #16

Merged
nicosuave merged 15 commits into
mainfrom
full-results
Nov 15, 2026
Merged

nicosuave merged 15 commits into
mainfrom
full-results

Conversation

@nicosuave

Copy link
Copy Markdown
Member

Adds a result cache so the notebook can show a statement's first rows at once and later page through, sort, filter, aggregate or export the rest without running the statement again.

select * from sqlnb_result_tee((select ...), id := ...) returns the statement's first rows and stores every row. It wraps the statement's result collector, so the statement runs on all its threads as it would without the tee and each thread stores the Arrow batches it produces, labelled with their executor batch; the footer lists them in the statement's order. In a local measurement the tee cost no more than the statement alone (0.077 s against 0.079 s). sqlnb_result(id, first_row, row_count, row_number) reads a window or the whole result back and reads only the batches and columns a query needs; sqlnb_results() lists results and sqlnb_result_drop(id) removes one.

Small results stay in memory. Larger ones are written as uncompressed Arrow IPC files under the user's cache directory, in a directory per process that is locked while the process runs and removed when it exits or, after a crash, by the next process. Limits default to 4 GiB per result and 8 GiB for the cache, results expire 24 hours after their last use, and nothing more is written once the cache's volume would drop below a free-space floor. A limit or a failure to store never fails or shortens the statement: the result keeps a prefix and reports why. A statement that fails or is interrupted leaves nothing behind. The MCP agent connection cannot use these functions.

The cache is written in Rust on the arrow-* 60.1.0 crates. On Apple platforms the extension now links CoreFoundation, which those crates need through chrono, so it still loads on its own. docs/results.md is the reference.

The 109 Rust tests, the macOS release build and its runtime smoke pass locally. test/sql/results.test has not been run under DuckDB's test runner locally, and Linux and Windows have not been built; this pull request's workflows are their first check. The README line conflicts with the Instant SQL pull request's, so whichever lands second needs main merged in.

Small results stay in memory; larger ones are written as Arrow IPC files in a
locked per-process session directory, renamed into place once complete. Limits
per result and for the whole cache keep a readable prefix, evicting the oldest
other results first. The C ABI in sidequery_http.h exchanges batches through the
Arrow C Data Interface.
sqlnb_result_tee passes a statement's first rows through in full chunks while
storing every row, and can stop reading once storing stops. sqlnb_results,
sqlnb_result and sqlnb_result_drop list, read back through DuckDB's Arrow scan,
and remove results. The MCP agent connection cannot reach any of them.
The Arrow crates read the local time zone's name (chrono's clock feature,
through iana-time-zone), which calls CoreFoundation on Apple platforms. The
extension only loaded after another extension that links the framework.
sqlnb_result_tee no longer limits the statement's last pipeline to one
thread. It wraps the statement's result collector: every thread stores the
rows it produces as Arrow batches keyed by their executor batch, the file's
footer lists them in that order, and only the first rows are passed on to
the client, exactly the first rows stored.

The runtime smoke test now loads SQLNB alone in a fresh process and checks
the stored order of a filtered scan, a sort, several Parquet files and a
join, and the prefix kept when a verified read stops at the limit.
sqlnb_result_tee takes ttl_seconds (24 hours unless named, NULL for as long
as DuckDB runs). A result expires that long after its statement finished or
sqlnb_result last read it, and is then gone like an evicted one. There is no
timer: binding or beginning a tee, listing, reading and dropping remove what
expired. sqlnb_results() reports last_used_ms and expires_ms.

The cache keeps 8 GiB unless a tee names another limit, and a tee whose limit
is below what is kept evicts the oldest results when it begins instead of
when it first stores a batch.

Nothing more is written to a result file once its volume would have less than
5 GiB free, or a tenth of a volume smaller than 50 GiB. Storing stops with
limit_reason 'disk-space'; the statement runs on.

The defaults now live with the cache in Rust and reach the function through
sidequery_result_defaults.
The thread that passes the head on created and released the wrapped
collector's shared state without the lock under which a thread leaving
the head tests it.
A statement that stops reading once storing has stopped now completes
its head first; threads that stopped earlier left the head with rows
from wherever each of them was. The documentation no longer promises
that every returned row is stored: both are the first rows of the
result, and a cache that stops storing keeps fewer.
Quack sends a result's last batches from an event it inserts while it
finalizes. The tee made the result readable as soon as that finalize
returned, so a statement that failed in the event, with the client gone
or the statement interrupted, left its result behind. The result is now
published from an event that runs after those.

Also say that the wrapped collector's own sink_state stays unset.
Every batch that ended before the lowest one left up to a head's worth
of rows waiting, so a slow or unstarted lowest batch let them grow with
the number of batches. Only the lowest ended batches now wait, up to the
one with which they fill the head; later batches keep no copy.
DuckDB does not interrupt a pipeline's combine and finalize: a cancel
that arrives then is raised afterwards, when the result has already
been made readable. Over Quack a client that stopped reading and
cancelled left the result of its failed statement listed and on disk.
The statement now drops the result it published when it ends in error.
Brings in the upload endpoint and the Instant SQL changes. The README lists
both the Instant SQL read policy and the result cache.
@nicosuave
nicosuave merged commit 3993b87 into main Nov 15, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant