⚡ Bolt: Remove eager list materialization in AST traversals - #178
⚡ Bolt: Remove eager list materialization in AST traversals#178tachyon-beep wants to merge 2 commits into
Conversation
💡 What: Changed AST traversal functions `_assignment_callee`, `_collect_return_paths`, and `_find_assignment_callee` to accept `Iterable[ast.AST]` instead of `list[ast.AST]`. Removed the `list()` wrapping around `ast.iter_child_nodes(node)` and `func_node.body` when passing them to these functions. 🎯 Why: Passing generators directly to recursive traversal functions prevents eager list materialization, eliminating unnecessary memory allocations and improving performance during AST analysis. 📊 Impact: Reduces memory usage and improves speed during deep AST traversals by iterating lazily over child nodes instead of instantiating new lists at every recursion level. 🔬 Measurement: Performance can be verified by profiling memory allocations during large codebase scans. Co-authored-by: tachyon-beep <544926+tachyon-beep@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
Pull request overview
This PR optimizes AST traversal in the scanner by switching key recursive helpers to accept Iterable[ast.AST] and by passing child-node iterators and function bodies directly, avoiding repeated list() materialization during deep traversals.
Changes:
- Updated
_assignment_calleeand_collect_return_pathsinvariable_level.pyto acceptIterable[ast.AST]and removed eagerlist()wrapping offunc_node.bodyandast.iter_child_nodes(...). - Updated
_find_assignment_calleeinflow_trace.pyto acceptIterable[ast.AST]and adjusted imports accordingly. - Added a Bolt note documenting the performance learning and preferred pattern for AST traversal APIs.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| src/wardline/scanner/taint/variable_level.py | Removes eager list materialization in recursive AST return/assignment traversal helpers by using Iterable and passing iterators directly. |
| src/wardline/scanner/flow_trace.py | Broadens _find_assignment_callee’s parameter type to Iterable and updates imports to match. |
| .jules/bolt.md | Documents the “avoid eager list() of AST generators” guidance for future changes. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
💡 What: Changed AST traversal functions `_assignment_callee`, `_collect_return_paths`, and `_find_assignment_callee` to accept `Iterable[ast.AST]` instead of `list[ast.AST]`. Removed the `list()` wrapping around `ast.iter_child_nodes(node)` and `func_node.body` when passing them to these functions. 🎯 Why: Passing generators directly to recursive traversal functions prevents eager list materialization, eliminating unnecessary memory allocations and improving performance during AST analysis. 📊 Impact: Reduces memory usage and improves speed during deep AST traversals by iterating lazily over child nodes instead of instantiating new lists at every recursion level. 🔬 Measurement: Performance can be verified by profiling memory allocations during large codebase scans. Co-authored-by: tachyon-beep <544926+tachyon-beep@users.noreply.github.com>
💡 What: Changed AST traversal functions
_assignment_callee,_collect_return_paths, and_find_assignment_calleeto acceptIterable[ast.AST]instead oflist[ast.AST]. Removed thelist()wrapping aroundast.iter_child_nodes(node)andfunc_node.bodywhen passing them to these functions.🎯 Why: Passing generators directly to recursive traversal functions prevents eager list materialization, eliminating unnecessary memory allocations and improving performance during AST analysis.
📊 Impact: Reduces memory usage and improves speed during deep AST traversals by iterating lazily over child nodes instead of instantiating new lists at every recursion level.
🔬 Measurement: Performance can be verified by profiling memory allocations during large codebase scans.
PR created automatically by Jules for task 17820923595346048879 started by @tachyon-beep