⚡ Bolt: [performance improvement] Avoid eager list materialization in AST traversals - #158
⚡ Bolt: [performance improvement] Avoid eager list materialization in AST traversals#158tachyon-beep wants to merge 2 commits into
Conversation
… AST traversals 💡 What: Modified `_collect_return_paths` and `_assignment_callee` to accept `Iterable[ast.AST]` instead of `list[ast.AST]`, and removed the `list()` wrapper around generator expressions like `ast.iter_child_nodes(node)` and property accesses like `func_node.body`. 🎯 Why: Eager list materialization forces Python to allocate a list in memory and iterate through the generator to fill it before passing it to the function. Since the function just iterates over the nodes anyway, this is a waste of memory and CPU cycles. 📊 Impact: Eliminates unnecessary list allocations during per-variable taint tracking, saving memory and avoiding an O(N) memory allocation step at every level of the AST during L2 processing. 🔬 Measurement: Observe memory usage during scans on large Python projects. Verify that tests continue to pass and no regressions have been introduced. Co-authored-by: tachyon-beep <544926+tachyon-beep@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
Pull request overview
This PR optimizes AST traversal in the taint scanner by removing unnecessary eager list materialization and widening helper function inputs from list[ast.AST] to Iterable[ast.AST], reducing allocations during recursive walks.
Changes:
- Updated
_collect_return_pathsand_assignment_calleeto acceptIterable[ast.AST]. - Removed
list(...)wrappers aroundfunc_node.bodyandast.iter_child_nodes(node)at call sites to avoid eager allocation.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
… AST traversals 💡 What: Modified `_collect_return_paths` and `_assignment_callee` to accept `Iterable[ast.AST]` instead of `list[ast.AST]`, and removed the `list()` wrapper around generator expressions like `ast.iter_child_nodes(node)` and property accesses like `func_node.body`. This also includes auto-formatting fixes that failed in CI. 🎯 Why: Eager list materialization forces Python to allocate a list in memory and iterate through the generator to fill it before passing it to the function. Since the function just iterates over the nodes anyway, this is a waste of memory and CPU cycles. 📊 Impact: Eliminates unnecessary list allocations during per-variable taint tracking, saving memory and avoiding an O(N) memory allocation step at every level of the AST during L2 processing. 🔬 Measurement: Observe memory usage during scans on large Python projects. Verify that tests continue to pass and no regressions have been introduced. Co-authored-by: tachyon-beep <544926+tachyon-beep@users.noreply.github.com>
💡 What: Modified
_collect_return_pathsand_assignment_calleeto acceptIterable[ast.AST]instead oflist[ast.AST], and removed thelist()wrapper around generator expressions likeast.iter_child_nodes(node)and property accesses likefunc_node.body.🎯 Why: Eager list materialization forces Python to allocate a list in memory and iterate through the generator to fill it before passing it to the function. Since the function just iterates over the nodes anyway, this is a waste of memory and CPU cycles.
📊 Impact: Eliminates unnecessary list allocations during per-variable taint tracking, saving memory and avoiding an O(N) memory allocation step at every level of the AST during L2 processing.
🔬 Measurement: Observe memory usage during scans on large Python projects. Verify that tests continue to pass and no regressions have been introduced.
PR created automatically by Jules for task 10599748386960697458 started by @tachyon-beep