fix: ray_execute and index attach never return NULL - #655
Merged
Merged
Conversation
Running cppcheck one file at a time -- which enables the cross-file checks
CI's parallel whole-tree run skips -- reported six ctunullpointer warnings.
Each has the same shape: a value that RAY_IS_ERR lets through as NULL,
then a dereference. Traced to the producers, two are real, reachable on
allocation failure:
- ray_execute: ray_execute_inner hands NULL back unchanged when a node,
a compaction or a merge fails to allocate -- the flat path's final
`return result` and the streaming path's seg_tbl / partial / merged
returns; only the streaming tail converted NULL to "oom". Callers test
RAY_IS_ERR, which is false for NULL: ray_lazy_materialize passes it on,
and (times <lazy> ...) dereferences it in loop_count (eval.c:1975).
ray_execute now converts NULL to an "oom" error at its single exit.
- prepare_attach_ex (idxop.c): a failed ray_cow, or a failed copy inside
ray_index_drop, returned NULL to all five callers (.idx.hash, dict,
part, built, plain attach), which test only RAY_IS_ERR and then
dereference (idxop.c:711). After ray_index_drop it also stored that
NULL into the caller's *vp. Both now return "oom" and leave *vp intact.
One is dead code made consistent: exec_group_v2 tested !g and then passed
g to exec_group_v2_run, which dereferences it (internal.h:650). Every
caller passes a graph; a NULL one now returns "nyi" like ray_execute.
The other three are false positives, inferred only from the NULL test
inside RAY_IS_ERR: ray_try_handle's handler (eval.c:301) comes from
ray_eval of a non-NULL expression; filter.c's gather only reaches `col`
for a new_cols[c] built from col->type; update_eval_on already converts
NULL to an error (query.c:14072). After this commit idxop.c:711 and
eval.c:1975 are false positives too, their producers no longer returning
NULL.
No regression test: every real path needs an allocation to fail and there
is no allocator fault injection to force one. make test 3956/3956.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Where these came from
With the cppcheck fix from #654, I ran cppcheck one file at a time. That turns on the cross-file checks CI's parallel whole-tree run skips, and it reported six
ctunullpointerwarnings. Each has the same shape: a value thatRAY_IS_ERRlets through as NULL (the macro is false for NULL), followed by a dereference. I traced each one to the code that produces the value.Real: reachable when an allocation fails
ray_executecan return NULL (src/ops/exec.c).ray_execute_innerhands NULL back unchanged when a node, a compaction or a merge fails to allocate: the flat path's finalreturn result, and the streaming path'sseg_tbl/partial/mergedreturns. Only the streaming path's tail converted NULL tooom. Callers testRAY_IS_ERR:ray_lazy_materializepasses the NULL on, and(times <lazy> ...)dereferences it inloop_count(eval.c:1975).ray_executenow converts NULL to anoomerror at its single exit, so all 124ray_lazy_materializecall sites are covered, not just the 49 that checked for NULL themselves.prepare_attach_excan return NULL (src/ops/idxop.c). A failedray_cow, or a failed copy insideray_index_drop, returned NULL to all five callers (.idx.hash, dict, part, built, plain attach). They test onlyRAY_IS_ERRand then dereference (idxop.c:711). Afterray_index_drop, it also stored that NULL into the caller's*vp, clobbering the caller's vector pointer. Both paths now returnoomand leave*vpintact.Dead code made consistent
exec_group_v2tested!g, then passedgtoexec_group_v2_run, which dereferences it (internal.h:650). Every caller passes a graph; a NULL one now returnsnyi, likeray_execute.False positives, left as they are
These are inferred only from the NULL test inside
RAY_IS_ERR:eval.c:301:ray_try_handle's handler comes fromray_evalof a non-NULL expression (tryis an arity-checked special form), or from the VM stack.filter.c:515: the gather only reachescolfor anew_cols[c]that was built fromcol->type.query.c:14072:update_eval_onalready converts NULL to an error.After this PR,
idxop.c:711andeval.c:1975are false positives too, because their producers no longer return NULL. Re-running cppcheck file by file on the changed files confirmsinternal.h:650is gone.Testing
make test: 3956/3956 pass, with no UBSan reports.