Skip to content

fix: surface Unknown for deep partial-eval patterns with missing ancestors - #194

Open
detail-app[bot] wants to merge 3 commits into
mainfrom
detail/bug-fix/fix-surface-unknown-for-deep-partial-eval-patterns-e8c255
Open

detail-app[bot] wants to merge 3 commits into
mainfrom
detail/bug-fix/fix-surface-unknown-for-deep-partial-eval-patterns-e8c255

Conversation

@detail-app

@detail-app detail-app Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Detail bug report: View on Detail

Change

Closes #171

Field-level partial evaluation dead-ended whenever a declared unknown pattern was deeper than the receiver chain at an attribute hop. PartialState.override() only consulted patterns where AttributePattern.matches(path) was true — and matches requires pattern.qualifiers.size() <= attribute.qualifiers.size() (shallower-or-equal only), so for a pattern like data.secret.deep against a hop like data or data.secret, override returned NONE. The SELECT/INDEX hop then raised a hard UNDECLARED (or NO_SUCH_KEY/NO_SUCH_FIELD when the container was present) before any overlap-based fallback could fire. The documented resolve-and-retry loop was unreachable: PartialResult.isComplete() was false while unknownAttributes() returned [] — no attribute to resolve, and even resolving the exact patterned leaf and retrying still threw.

The change adds an overlap-based fallback (PartialState.unknownIntersect) at the SELECT and INDEX hops that mirrors the bidirectional semantics already used by PartialState.consume and PartialState.containsUnknown:

  • any pattern that intersects the hop path returns an Unknown rooted at the longest common prefix of the hop path and the pattern qualifiers (the hop path itself when the pattern is deeper, the pattern granularity when shallower-or-equal — preserving the existing matches-rooted form);
  • a resolved ancestor that has supplied (or supplies) the receiver still steers the normal selector read, so the existing hard-error semantics for resolved-then-missing descendants are intact (data = {} resolution + data.secret still raises NO_SUCH_KEY);
  • non-intersecting siblings keep their existing UNDECLARED/NO_SUCH_KEY/NO_SUCH_FIELD outcome (data.known with pattern data.secret keeps raising UNDECLARED).

This matches cel-go and cel-java, where pattern matching is overlap-based in either direction and runs against the resolved attribute path before the activation is consulted. The optional ?. / [?...] null-coalescing path is unchanged; consume already converts the optional result to an Unknown when a pattern intersects the path and no resolution covers it, and the resolved-ancestor check in the new fallback keeps the OptionalValue.none() outcome intact when a parent has been resolved.

Validation

  • New regression test DeepPatternReproTest (11 cases) pins every scenario from the bug report — deep pattern + missing ancestors (data.secret, data['secret'], data.secret.deep, has(data.secret)), present container + missing intermediate key, exact-leaf and ancestor resolution retries, wildcard-deep patterns, reverse-depth shallow pattern (regression control), non-intersecting sibling UNDECLARED, resolved-ancestor NO_SUCH_KEY, comprehension merging, INDEX via data['secret']['deep'], and optional access with a resolved ancestor. bazel test --config=local //src/test/java/dev/trufflecel:deepPatternRepro — PASSED (11 tests).
  • Existing PartialEvaluationTest (13 cases) — PASSED.
  • Broader regression sweep: runtime, checker, parser, activation, borrowedBindings, optionalValue, foreignOrdinalBug, pojo, preparedField, canonicalAst, reproLeak, mapSnapshot, lazyBinding, dataIntegration, protobufIntegration, jsonIntegration, extensions, libraryPolicy, functionOverload, lateFunction, environmentControls, evaluationControls, diagnostics, costEstimator, astMacro, interop, validation, cse, negativeMatrix, policyConformance, pojoProcessor — all PASSED locally.
  • bazel test --config=local //src/test/java/dev/trufflecel:conformance — PASSED (2,771-case cel-spec corpus).
  • bazel build --config=local //:trufflecel //:trufflecel-protobuf //:trufflecel-json //:trufflecel-processor — clean.
  • bazel run --config=local //tools:format -- --check — clean; new test and edited production files conform.
  • bazel run --config=local //bazel/gazelle:check — no drift; the new java_test entry is generated.
  • Re-ran formatter-apply, then re-verified the partial-eval and conformance suites — still PASS.

Two unrelated targets (traceAsync, instrumentation) fail in this sandbox on the unchanged baseline too: they require the GraalVM engine.CompilationFailureAction option, which is unavailable when polyglot.engine.Compilation and the native Truffle attach library are not installed (no access to /root/.cache/org.graalvm.polyglot). This is environmental and not affected by the change.

Cross-implementation comparison with cel-go/cel-java (the reference behavior the fix mirrors) is cited directly in the issue as evidence; the TruffleCEL DeepPatternReproTest cases are the equivalent pin for this implementation. bazel coverage --config=local //src/test/... and bazel run --config=local //tools:coverage (99.8% floor) and the Linux native guest-JIT/blocklist smoke (//benchmarks:jit_smoke) were not run in this environment; CI will run them as part of the full required gates.


Automatic Fixes PRs can be configured here.

? OptionalValue.of(override)
: override;
if (a == PartialState.MISSING) throw missingAttribute(parent);
if (a == PartialState.MISSING) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reference equality is intentional here: PartialState.MISSING/NONE are identity-only static final Object sentinels, so .equals() would be a no-op at best and NPE-prone at worst (the receiver can be null, e.g. when an earlier read threw and was captured). This matches the existing convention throughout ExprNode (e.g. override != PartialState.NONE on the line above). Not changing.

// this read path turns the hard error into an Unknown for resolve-and-retry; a
// resolution that supplies (or has supplied) an ancestor still raises the error.
Object intersect = ctx.attributeIntersect(path);
if (intersect != PartialState.NONE) yield intersect;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reference equality is intentional here: PartialState.MISSING/NONE are identity-only static final Object sentinels, so .equals() would be a no-op at best and NPE-prone at worst (the receiver can be null, e.g. when an earlier read threw and was captured). This matches the existing convention throughout ExprNode (e.g. override != PartialState.NONE on the line above). Not changing.

? OptionalValue.of(override)
: override;
if (operand == PartialState.MISSING) throw missingAttribute(parent);
if (operand == PartialState.MISSING) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reference equality is intentional here: PartialState.MISSING/NONE are identity-only static final Object sentinels, so .equals() would be a no-op at best and NPE-prone at worst (the receiver can be null, e.g. when an earlier read threw and was captured). This matches the existing convention throughout ExprNode (e.g. override != PartialState.NONE on the line above). Not changing.

// surface an Unknown for resolve-and-retry instead of a hard NO_SUCH_KEY; a resolution
// that supplies (or has supplied) an ancestor steers the existing error path.
Object intersect = ctx.attributeIntersect(path);
if (intersect != PartialState.NONE) return intersect;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reference equality is intentional here: PartialState.MISSING/NONE are identity-only static final Object sentinels, so .equals() would be a no-op at best and NPE-prone at worst (the receiver can be null, e.g. when an earlier read threw and was captured). This matches the existing convention throughout ExprNode (e.g. override != PartialState.NONE on the line above). Not changing.

if (!type.fields().containsKey(name))
if (!type.fields().containsKey(name)) {
Object intersect = ctx.attributeIntersect(path);
if (intersect != PartialState.NONE) return intersect;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reference equality is intentional here: PartialState.MISSING/NONE are identity-only static final Object sentinels, so .equals() would be a no-op at best and NPE-prone at worst (the receiver can be null, e.g. when an earlier read threw and was captured). This matches the existing convention throughout ExprNode (e.g. override != PartialState.NONE on the line above). Not changing.

* cel-java, where the pattern check runs against the resolved attribute path before the activation
* is consulted.
*/
public class DeepPatternReproTest {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a non-gating trufflecheck baseline finding: per AGENTS.md, trufflecheck is not a default gate while docs/TRUFFLECHECK_BASELINE.md has open findings, and the rest of the test suite's files aren't @NullMarked either, so annotating only this file would be inconsistent. Not changing.

@codspeed

codspeed Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 21 untouched benchmarks


Comparing detail/bug-fix/fix-surface-unknown-for-deep-partial-eval-patterns-e8c255 (f7300ad) with main (a31f1c5)

Open in CodSpeed

@codecov

codecov Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@datadog-official

datadog-official Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Pipelines

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: f7300ad | Docs | Give us feedback!

…aths

attributeIntersect/unknownIntersect only ever returns NONE or an Unknown, so
the optional-lifting ternaries added to the SELECT/INDEX missing-receiver
blocks were unreachable. Removing them restores the 99.8% coverage floor (the
dead branches were never exercised) and shrinks the inlined Graal graph for
SELECT/INDEX nodes, bringing the representative smoke benchmark back under the
10000-node compilation limit. Add regression tests for the two remaining
intersect paths: a missing receiver with an unknown index key, and a deep
pattern on a runtime object's missing field reached via dyn().
// `attributeIntersect` only ever returns NONE or an Unknown, so the optional-lifting form
// used for `override` above is not needed here.
Object intersect = ctx.attributeIntersect(path);
if (intersect != PartialState.NONE) yield intersect;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reference equality is intentional here: PartialState.NONE is an identity-only static final Object sentinel, and per the comment on the preceding lines ctx.attributeIntersect(path) only ever returns NONE or an Unknown, so .equals() would be a no-op at best. This matches the existing convention throughout ExprNode (e.g. intersect != PartialState.NONE on the line above and the same pattern at the other call sites). Not changing.

// surface the existing error. `attributeIntersect` only ever returns NONE or an Unknown,
// so the optional-lifting form used for `override` above is not needed here.
Object intersect = ctx.attributeIntersect(path);
if (intersect != PartialState.NONE) return intersect;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reference equality is intentional here: PartialState.NONE is an identity-only static final Object sentinel, and per the comment on the preceding lines ctx.attributeIntersect(path) only ever returns NONE or an Unknown, so .equals() would be a no-op at best. This matches the existing convention throughout ExprNode (e.g. intersect != PartialState.NONE on the line above and the same pattern at the other call sites). Not changing.

Add two regression tests for paths the PR introduced but CI's coverage
gate flagged as uncovered: multiple deeper-than-the-hop patterns
intersecting the same SELECT path (exercises the multi-pattern merge in
PartialState.unknownIntersect), and a non-partial read of a missing map
key via the options-bearing evaluate entry (exercises the
evaluation.partial == null arm of EvalContext.attributeIntersect, which
confirms the new error-path call is a no-op outside partial evaluation).
@Test
public void nonPartialMissingKeyStillThrowsNoSuchKey() {
var env = Cel.newBuilder().variable("m", CelType.map(CelType.STRING, CelType.INT)).build();
var thrown =

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a non-gating trufflecheck baseline finding: per AGENTS.md/docs/TRUFFLECHECK_BASELINE.md, AssertThrowsMinimizer is a P4 WARN baseline finding (118 open), not enforced by --config=trufflecheck. The lambda body is the test itself — the comment above the test explains why the options-bearing evaluate(Map, EvaluationOptions) entry must be used (only it configures an EvaluationState with non-null evaluation and null partial, which is the arm being covered). Extracting it into a helper would obscure the only thing the test asserts. Not changing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Detail Bug] Partial evaluation returns hard UNDECLARED/NO_SUCH_KEY instead of Unknown for deep unknown patterns with missing ancestors

2 participants