perf(call-graph): walk the raw AST when indexing call expressions - #142
Conversation
`indexCallExpressions` traversed with ts-morph's `forEachDescendant`, which wraps every visited node in a JS object and caches it on the SourceFile for the lifetime of the program. Indexing call sites therefore materialised the entire AST of every source file, not just the call-like nodes it keeps. Recurse the raw compiler nodes instead and wrap only the matches. Measured on vscode/src (6,758 files, 470,973 indexed call sites): the phase drops from +3.23GB to +2.52GB and runs 1.1s faster, with byte-identical analysis.json output. A differential test pins the raw walk against the wrapper walk it replaced.
|
Marking this draft. The end-to-end A/B does not support the memory claim in the description. Same machine, same input, same flags (
0.04 GB apart is noise, and this branch is ~49s slower. The phase-level reductions I measured are real — the call-site index goes from +3.23 GB to The 27.0 GB target in #141's definition of done is also wrong. It came from an earlier run whose What still stands on its own:
Next step is to attribute the two commits separately rather than defend them together. Not merging |
8a58686 to
46f9357
Compare
|
Attribution run done. Reduced this PR to the raw walk alone.
The interleave commit costs 0.90 GB rather than saving anything, so it is dropped from this branch (kept on |
Closes #141.
indexCallExpressionstraversed with ts-morph'sforEachDescendant, which wraps every visitednode in a JS object and caches it on the
SourceFilefor the lifetime of the program. Indexingcall sites therefore kept the entire AST of every file resident, for the sake of the ~471k
call-like nodes it actually keeps. This recurses the raw compiler nodes and wraps only the matches.
Measured
End-to-end on
vscode/src(6,758 files, 470,973 call sites), same machine, same flags(
--no-build -a 4 --graphs cfg,dfg,pdg,sdg):origin/main−0.94 GB, about 3.2% of peak. The index phase itself goes from +3.23 GB to +2.52 GB.
Output is unchanged:
analysis.jsonis byte-identical at-a 4with--graphs cfg,dfg,pdg,sdgon
dataflow-app,sample-appandmulti-tsconfig-app, and thevscode/srcrun above producesthe same 6,758 modules and 1,053,108 call-graph edges as
origin/main.A differential test pins the raw walk against the wrapper walk it replaces. It was mutation-checked:
dropping tagged templates from the kind filter, and dropping the recursion, each make it fail.
Scope, and what was dropped
This PR originally also carried a per-program "interleave solve, extract, release" commit. Measured
separately, that commit added 0.90 GB — 28.75 GB with the raw walk alone, 29.65 GB with both —
so it has been dropped rather than merged on its description. It is preserved on
perf/interleave-disposeif anyone wants to take it further.What this does not do
It does not make whole-vscode
-a 4fit. Per #141, 21.33 GB is already committed before theinterprocedural phase begins, and that phase's entire retained state is 0.48 GB. This is a 0.94 GB
reduction against a ~29.7 GB peak, not a change to the ceiling. Whole-vscode
-a 3completes today(19m41s, 28.4 GB);
-a 4needs #112 Step 3 or 4.Wall time across single runs was 9m37s (main), 10m34s (this branch) — within the run-to-run spread
I saw on this machine, and not something I would claim either direction from one sample each.