Skip to content

fix: ray_execute and index attach never return NULL - #655

Merged
singaraiona merged 1 commit into
devfrom
fix/execute-never-null
Sep 30, 2026
Merged

singaraiona merged 1 commit into
devfrom
fix/execute-never-null

Conversation

@singaraiona

Copy link
Copy Markdown
Collaborator

Where these came from

With the cppcheck fix from #654, I ran cppcheck one file at a time. That turns on the cross-file checks CI's parallel whole-tree run skips, and it reported six ctunullpointer warnings. Each has the same shape: a value that RAY_IS_ERR lets through as NULL (the macro is false for NULL), followed by a dereference. I traced each one to the code that produces the value.

Real: reachable when an allocation fails

ray_execute can return NULL (src/ops/exec.c). ray_execute_inner hands NULL back unchanged when a node, a compaction or a merge fails to allocate: the flat path's final return result, and the streaming path's seg_tbl / partial / merged returns. Only the streaming path's tail converted NULL to oom. Callers test RAY_IS_ERR: ray_lazy_materialize passes the NULL on, and (times <lazy> ...) dereferences it in loop_count (eval.c:1975). ray_execute now converts NULL to an oom error at its single exit, so all 124 ray_lazy_materialize call sites are covered, not just the 49 that checked for NULL themselves.

prepare_attach_ex can return NULL (src/ops/idxop.c). A failed ray_cow, or a failed copy inside ray_index_drop, returned NULL to all five callers (.idx.hash, dict, part, built, plain attach). They test only RAY_IS_ERR and then dereference (idxop.c:711). After ray_index_drop, it also stored that NULL into the caller's *vp, clobbering the caller's vector pointer. Both paths now return oom and leave *vp intact.

Dead code made consistent

exec_group_v2 tested !g, then passed g to exec_group_v2_run, which dereferences it (internal.h:650). Every caller passes a graph; a NULL one now returns nyi, like ray_execute.

False positives, left as they are

These are inferred only from the NULL test inside RAY_IS_ERR:

  • eval.c:301: ray_try_handle's handler comes from ray_eval of a non-NULL expression (try is an arity-checked special form), or from the VM stack.
  • filter.c:515: the gather only reaches col for a new_cols[c] that was built from col->type.
  • query.c:14072: update_eval_on already converts NULL to an error.

After this PR, idxop.c:711 and eval.c:1975 are false positives too, because their producers no longer return NULL. Re-running cppcheck file by file on the changed files confirms internal.h:650 is gone.

Testing

  • make test: 3956/3956 pass, with no UBSan reports.
  • No regression test. Every real path needs an allocation to fail, and the allocator has no fault-injection hook to force one. Adding one just for these one-line guards seemed out of proportion.

Running cppcheck one file at a time -- which enables the cross-file checks
CI's parallel whole-tree run skips -- reported six ctunullpointer warnings.
Each has the same shape: a value that RAY_IS_ERR lets through as NULL,
then a dereference.  Traced to the producers, two are real, reachable on
allocation failure:

  - ray_execute: ray_execute_inner hands NULL back unchanged when a node,
    a compaction or a merge fails to allocate -- the flat path's final
    `return result` and the streaming path's seg_tbl / partial / merged
    returns; only the streaming tail converted NULL to "oom".  Callers test
    RAY_IS_ERR, which is false for NULL: ray_lazy_materialize passes it on,
    and (times <lazy> ...) dereferences it in loop_count (eval.c:1975).
    ray_execute now converts NULL to an "oom" error at its single exit.

  - prepare_attach_ex (idxop.c): a failed ray_cow, or a failed copy inside
    ray_index_drop, returned NULL to all five callers (.idx.hash, dict,
    part, built, plain attach), which test only RAY_IS_ERR and then
    dereference (idxop.c:711).  After ray_index_drop it also stored that
    NULL into the caller's *vp.  Both now return "oom" and leave *vp intact.

One is dead code made consistent: exec_group_v2 tested !g and then passed
g to exec_group_v2_run, which dereferences it (internal.h:650).  Every
caller passes a graph; a NULL one now returns "nyi" like ray_execute.

The other three are false positives, inferred only from the NULL test
inside RAY_IS_ERR: ray_try_handle's handler (eval.c:301) comes from
ray_eval of a non-NULL expression; filter.c's gather only reaches `col`
for a new_cols[c] built from col->type; update_eval_on already converts
NULL to an error (query.c:14072).  After this commit idxop.c:711 and
eval.c:1975 are false positives too, their producers no longer returning
NULL.

No regression test: every real path needs an allocation to fail and there
is no allocator fault injection to force one.  make test 3956/3956.
@singaraiona
singaraiona merged commit 2e2a4d6 into dev Sep 30, 2026
10 checks passed
@singaraiona
singaraiona deleted the fix/execute-never-null branch September 30, 2026 12:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant