Add Queryable nodes for the new LINQ operators - #15
Open
davidanthoff wants to merge 3 commits into
Open
davidanthoff wants to merge 3 commits into
davidanthoff wants to merge 3 commits into
Conversation
get_source and _collect_nodes! needed a method per node type that combines two sources, because those carry `outer`/`inner` rather than `source`. With outer joins, seven set operators and zip arriving, that is eleven more pairs of near-identical methods. Introduce QueryableBinary instead, and subtype QueryableJoin and QueryableGroupJoin under it. The tree walk now has one method covering every two-input node, and a new one only has to pick the right supertype. QueryableBinary <: Queryable and the concrete types are unchanged, so `node isa QueryableJoin` and anything dispatching on Queryable still behave exactly as before. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds a node per operator introduced in QueryOperators 1.2, so a backend sees them in the query tree rather than having the in-memory implementation run behind its back: the outer joins, the seven set operators, the four partitioning operators, reverse/shuffle/index, append/prepend/zip, count_by/aggregate_by/chunk, and of_type/cast. Each gets a describe_node method so @queryplan() renders it. order and order_descending add no node — they lower to orderby with an identity key selector, so thenby can still follow them and a backend can recognise the identity selector and sort by every column. The plain and _by forms of the set operators share a node, with a `nothing` key selector marking the plain form; zip does the same with its result selector. Terminal operators needed new machinery. The tree had no notion of an operator that returns a value, and QueryOperators.count had no method taking a Queryable at all, so a query ending in @count() failed outright. QueryableScalar now captures one, and execute_scalar dispatches on the root source type so a backend can specialise _execute_scalar and translate the operator; the fallback materialises the source and delegates to the in-memory implementation, which every backend gets for free. count is fixed as a side effect, and a regression test pins it. Requires QueryOperators 1.2, which is not registered yet, so CI here stays red until it is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
davidanthoff
force-pushed
the
add-missing-linq-operators
branch
from
September 17, 2026 05:35
b5c6379 to
f781a67
Compare
A backend that translates only some terminal operators needs a way to hand the rest back. Splitting _execute_scalar_fallback out of the QueryableSource method gives it one, so its coverage can stay partial without affecting correctness. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second of four PRs adding the LINQ operators Query.jl is missing. Depends on QueryOperators.jl#56.
CI will stay red until QueryOperators 1.2.0 is registered. The
[compat]bound is an ordinaryQueryOperators = "1.2"; no feature guards. Locally the full suite passes (22 items, 14 new) against the QueryOperators branch.What it does
Adds a
Queryablenode for each operator introduced in QueryOperators 1.2, so a backend sees them in the query tree instead of having the in-memory implementation run behind its back — outer joins, the seven set operators, the four partitioning operators,reverse/shuffle/index,append/prepend/zip,count_by/aggregate_by/chunk, andof_type/cast. Each gets adescribe_nodemethod so@queryplan()renders it rather than falling through to the type-name fallback (a test asserts that).Three things worth reviewing
A
QueryableBinaryabstract type.get_sourceand_collect_nodes!needed a method per node that combines two sources, since those carryouter/innerrather thansource. Outer joins, set operators andzipwould have added eleven more pairs of near-identical methods.QueryableJoinandQueryableGroupJoinare now subtypes ofQueryableBinary, the tree walk has one method covering every two-input node, and a new node only has to pick the right supertype.QueryableBinary <: Queryableand the concrete types are unchanged, sonode isa QueryableJoinand anything dispatching onQueryablebehave exactly as before — QueryDuckDB needs no change for this.Terminal-operator support, and a bug it fixes. The tree had no notion of an operator that returns a value, and
QueryOperators.counthad no method taking aQueryableat all — sodf |> @duckdb() |> @count()failed outright today.QueryableScalarnow captures one, andexecute_scalardispatches on the root source type:A backend specialises
_execute_scalaron its own source type to push the operator down; the fallback materialises and delegates to the in-memory implementation, so every backend is correct for free and push-down is a pure optimisation.countis fixed as a side effect and a regression test pins it. QueryDuckDB will specialise this in the fourth PR.orderadds no node. It lowers toorderbywith an identity key selector, sothenbycan still follow it and a backend can recognise the identity selector via theis_identity_lambdahelper QueryDuckDB already has, and emitORDER BY ALL.Smaller: the plain and
_byforms of the set operators share a node, with anothingkey selector marking the plain form;zipdoes the same with its result selector.Tests call
QueryOperators.*directly rather than through Query macros, since those land in the Query.jl PR.🤖 Generated with Claude Code