[prrao87]

Why

MATCH (a)-[:FOLLOWS]->(b) OPTIONAL MATCH (b)-[:HAS_INTEREST]->(i) RETURN count(*) kept the join plan: the optional hop was expanded from the distinct b, left-joined back onto every first-hop row, and only then counted. The two-hop count rewrite can count it from degrees like an ordinary second hop. This is the other half of the "undirected hops and OPTIONAL MATCH" item in #78. Builds on #132 (merged), as both change the same rewrite.

What changed

• When the second hop is an OPTIONAL MATCH from the first hop's b, both hops are grouped per b as before and the optional side is left-joined. A b without a match still has one row per first-hop edge, with nulls: the count is Σ n1(b) × coalesce(n2(b), 1), or coalesce(n2(b), 0) for count(i) or count(r2), which skip nulls.
• The optional clause's own WHERE filters the second hop's edges, including conditions on b alone, since a b it rejects still keeps its null row. The first clause's WHERE applies to the first hop as before.
• No pairs are subtracted: the rule that a path uses each edge once applies within one clause, so the optional hop may walk the first hop's edge again, as on the join path.
• Anything else keeps the usual plan: a filter after the optional match, an optional pattern seeded from more than b, or a hop that doesn't qualify on its own.
• The CSR guide and the optimizer internals page describe the optional case.

Validation

• cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and cargo fmt --all --check pass. make test (1710 passed) and make lint pass, as do zensical build --strict and typos. CI is pending.

• New tests check the count against the join plan and against hand-counted answers: count(*), count(a) and count(c) (which skips the null rows), the optional clause's WHERE on its far end and on b alone, the first clause's WHERE, and an optional hop over the first hop's own type, which may reuse its edge.

• graph-benchmark (100K Person, 2.4M FOLLOWS), median ms as in perf(planner): count same-type two-hop paths that share an end #131; Apple M5, 24 GB, macOS 26.6.2; release wheels built from each commit and loaded with uv run --with. Answers are equal across builds and engines. "Before" is the undirected-hops branch this stacks on.

  Query	Answer	before, join	before, CSR	branch join	branch CSR
  MATCH (a)-[:FOLLOWS]->(b) OPTIONAL MATCH (b)-[:HAS_INTEREST]->(i) RETURN count(*)	6,086,831	27.6	30.5	7.8	7.9
  MATCH (a)-[:FOLLOWS]->(b) OPTIONAL MATCH (b)-[:FOLLOWS]->(c) RETURN count(c)	58,431,994	64.3	138.4	15.7	12.5
  q8, the chain (control, unchanged)	58,431,994	19.6	13.2	19.5	13.2

• Both suites, timed for the whole stack (this branch, with perf(planner): count same-type two-hop paths that share an end #131 and perf(planner): count two-hop paths over undirected hops #132) against main with the CONTRIBUTING.md harness: GDC SF1, graph-benchmark-ldbc 3afadb9 and graph-benchmark-ldbc-complex fc7b44c, both modes, alternating main and the stack twice each. Every test passes. Every query's physical plan is the same as on main (captured through explain()), apart from the order in which IC3 prints the two filters of its inline {name, type} map, which changes between builds.

  Suite	Mode	Geomean stack/main	Median gap between the two main runs
  graph-benchmark-ldbc	memory	1.016	1.7%
  graph-benchmark-ldbc	lazy	0.999	0.7%
  graph-benchmark-ldbc-complex	memory	1.019	0.6%
  graph-benchmark-ldbc-complex	lazy	0.996	1.6%

  Two rows are beyond the run gap:

  • IC12 (+13% in memory): it swings between about 17 and 20 ms per process on every build. Three more alternating runs per build measured 17.0 to 19.8 ms on main and 19.1 to 20.0 ms here, with the shared-end and undirected builds in the same range.
  • Custom q20 (+10.5% memory, +9.6% lazy): MATCH (c:Comment) WHERE c.content CONTAINS $fragment AND c.length > $min RETURN count(c.id), one scan and filter with no hop, which this change returns from at its first check. It is reproducible over three more alternating runs: 3.86 to 3.93 ms here, against 3.47 to 3.51 ms on main and on the shared-end and undirected builds. Timed on its own in one process, planning (explain()) takes 0.27 ms on every build, and the 0.38 ms difference is all in executing the same plan, in code this change doesn't touch. I read it as code placement in this build: without LTO, generic Arrow and DataFusion kernels are compiled into this crate, and any change to it can move them. I haven't confirmed that with a control build.

Every query, both suites and modes

graph-benchmark-ldbc: memory (ms, mean of two runs)

Query	main	this stack	change	main run gap	stack run gap
query1	0.66	0.65	-1.3%	15.6%	8.7%
query2	0.54	0.55	+2.1%	0.3%	3.7%
query3	0.62	0.64	+2.5%	0.4%	2.9%
query4	0.49	0.50	+2.0%	0.7%	3.6%
query5	1.09	1.06	-2.3%	1.3%	0.6%
query6	0.43	0.43	+2.0%	0.0%	4.4%
query7	1.01	1.03	+2.1%	1.5%	2.9%
query8	0.63	0.64	+1.4%	0.4%	2.3%
query9	0.80	0.81	+2.1%	1.9%	2.8%
query10	0.71	0.72	+1.7%	2.0%	3.1%
query11	1.52	1.54	+1.1%	1.7%	1.2%
query12	0.74	0.75	+1.1%	0.4%	2.6%
query13	0.78	0.80	+2.7%	1.7%	4.6%
query14	0.52	0.53	+1.5%	2.1%	5.4%
query15	0.55	0.56	+1.5%	2.2%	4.0%
query16	0.66	0.67	+1.5%	2.0%	3.8%
query17	0.90	0.91	+1.0%	1.7%	1.9%
query18	0.69	0.70	+0.9%	1.8%	3.5%
query19	1.02	1.03	+1.0%	1.4%	2.6%
query20	3.48	3.85	+10.5%	0.7%	1.0%
query21	0.75	0.76	+1.4%	1.2%	1.0%
query22	0.71	0.72	+1.2%	1.5%	2.4%
query23	0.72	0.73	+1.2%	2.5%	3.0%
query24	0.69	0.69	+0.9%	2.3%	2.0%
query25	0.79	0.79	+1.1%	1.9%	1.3%
query26	0.73	0.74	+1.3%	2.6%	1.8%
query27	0.86	0.87	+1.8%	2.3%	3.1%
query28	1.38	1.43	+3.9%	2.0%	1.9%
query29	0.70	0.69	-1.5%	8.0%	1.7%
query30	41.59	42.78	+2.9%	1.5%	1.8%

Geomean stack/main 1.016; median main run gap 1.7%

graph-benchmark-ldbc: lazy (ms, mean of two runs)

Query	main	this stack	change	main run gap	stack run gap
query1	1.17	1.15	-1.5%	3.0%	2.1%
query2	1.41	1.40	-0.4%	0.5%	3.1%
query3	0.81	0.80	-1.6%	4.2%	0.4%
query4	0.79	0.78	-0.6%	0.5%	2.9%
query5	3.68	3.58	-2.9%	1.6%	0.7%
query6	0.74	0.73	-0.3%	0.0%	2.7%
query7	1.10	1.08	-1.0%	1.0%	2.6%
query8	1.74	1.76	+1.0%	2.5%	1.4%
query9	1.34	1.34	-0.4%	0.1%	1.8%
query10	0.93	0.93	-0.7%	0.6%	3.2%
query11	1.99	1.97	-1.0%	0.3%	2.2%
query12	6.86	6.45	-6.0%	3.2%	2.8%
query13	1.06	1.05	-0.7%	0.8%	2.4%
query14	0.69	0.69	-0.2%	0.2%	3.6%
query15	0.76	0.76	-0.1%	0.4%	4.2%
query16	1.11	1.10	-0.6%	0.7%	2.6%
query17	1.39	1.39	-0.2%	0.4%	1.9%
query18	1.14	1.14	+0.6%	0.6%	2.4%
query19	1.51	1.52	+0.3%	1.9%	2.8%
query20	23.75	26.02	+9.6%	0.3%	1.1%
query21	0.84	0.84	-0.1%	1.4%	1.0%
query22	0.92	0.93	+0.6%	0.3%	1.8%
query23	1.23	1.23	+0.1%	0.7%	2.0%
query24	1.06	1.06	+0.3%	1.2%	1.5%
query25	1.20	1.19	-0.4%	1.5%	1.6%
query26	1.06	1.07	+0.6%	0.4%	1.9%
query27	1.29	1.30	+0.5%	1.4%	2.6%
query28	2.04	2.04	+0.0%	0.3%	0.8%
query29	1.04	1.02	-1.5%	2.3%	0.1%
query30	41.67	43.03	+3.3%	0.4%	0.5%

Geomean stack/main 0.999; median main run gap 0.7%

graph-benchmark-ldbc-complex: memory (ms, mean of two runs)

Query	main	this stack	change	main run gap	stack run gap
query1	23.22	23.12	-0.5%	0.1%	0.2%
query2	2.50	2.51	+0.4%	0.6%	2.8%
query3	25.39	26.29	+3.6%	2.2%	9.8%
query4	5.78	5.89	+1.8%	0.2%	3.1%
query5	74.59	75.04	+0.6%	0.5%	2.4%
query6	22.23	22.19	-0.2%	1.0%	1.8%
query7	9.77	9.89	+1.2%	1.6%	2.3%
query8	2.26	2.29	+1.0%	1.6%	1.1%
query9	38.37	38.06	-0.8%	0.4%	2.5%
query10	20.29	20.87	+2.9%	0.1%	5.6%
query11	2.99	3.08	+2.9%	1.0%	2.0%
query12	17.56	19.89	+13.3%	0.8%	0.7%
query13	1.10	1.11	+0.5%	0.3%	0.0%
query14	48.79	49.06	+0.5%	0.3%	1.6%

Geomean stack/main 1.019; median main run gap 0.6%

graph-benchmark-ldbc-complex: lazy (ms, mean of two runs)

Query	main	this stack	change	main run gap	stack run gap
query1	31.98	31.66	-1.0%	0.7%	0.1%
query2	25.44	25.25	-0.8%	1.1%	3.7%
query3	166.85	160.06	-4.1%	6.5%	2.7%
query4	12.77	12.62	-1.2%	3.3%	0.3%
query5	112.16	114.66	+2.2%	4.1%	0.8%
query6	22.64	22.87	+1.0%	3.6%	5.5%
query7	14.73	14.42	-2.1%	0.6%	2.4%
query8	3.33	3.40	+2.3%	1.6%	9.1%
query9	315.79	305.60	-3.2%	0.8%	0.7%
query10	29.18	28.58	-2.0%	1.0%	1.0%
query11	5.39	5.40	+0.3%	1.8%	0.1%
query12	21.87	22.82	+4.3%	8.2%	1.5%
query13	1.43	1.44	+0.2%	0.2%	0.4%
query14	50.34	49.98	-0.7%	1.3%	0.0%

Geomean stack/main 0.996; median main run gap 1.6%

Part of #78.

🤖 Generated with Claude Code

[cloudflare-workers-and-pages]

Deploying lance-csr with    Cloudflare Pages

Latest commit:	a312a10
Status:	⚡️  Build in progress...

View logs