Problem
Whole-repository analysis holds every program's ts-morph Project at once. On vscode that is 96
programs and it is what kills -a 4 (28.0-31.7 GB, exit 133). There is no way to analyse one
program at a time, which is the precondition for
the sharded, two-wave design (#112 Step 4).
Scope boundary
Unit 1 of the spec: the program selector only. Emits a normal, complete analysis.json for the
selected program(s) — it does not union shards (unit 3) or stitch cross-shard dataflow (unit 4).
Not in scope: IR persistence (unit 2), the union (unit 3), wave 2 (unit 4).
Goals
Caveats and known risks
The reassignment bug is the whole risk. ownerProgram falls back to the root program for any
file no scope contains. Filtering the spec list before assignment would silently pull files owned
by deeper tsconfigs into a selected ancestor, compiling them under the wrong config — wrong module
resolution, wrong paths aliases, wrong lib. Ownership must be computed globally and filtered
after. This needs a direct test, not just an end-to-end one.
Measured shard distribution on vscode (13,370 files, 96 non-empty programs):
| shard |
files |
share |
src |
8,966 |
67.1% |
extensions/copilot |
2,659 |
19.9% |
| remaining 94 |
<= 226 each |
~13% |
So this does not deliver a low peak: the largest shard is the same tree that already measures
28.75 GB standalone at -a 4. What it delivers is that the peak is bounded by ONE program instead
of all 96 — which is the difference between completing and being killed. There is no headroom
beyond that, and src cannot be split further without breaking id stability.
Definition of done
- Selecting every program reproduces the whole-repo run's module set exactly
- A file owned by a deeper unselected tsconfig is absent from the output, not reassigned
bun test green; --input/--app-name produce ids identical to the unsharded run
- Peak RSS measured on vscode's
src shard, reported against the 28.75 GB standalone baseline
Problem
Whole-repository analysis holds every program's ts-morph
Projectat once. On vscode that is 96programs and it is what kills
-a 4(28.0-31.7 GB, exit 133). There is no way to analyse oneprogram at a time, which is the precondition for
the sharded, two-wave design (#112 Step 4).
Scope boundary
Unit 1 of the spec: the program selector only. Emits a normal, complete
analysis.jsonfor theselected program(s) — it does not union shards (unit 3) or stitch cross-shard dataflow (unit 4).
Not in scope: IR persistence (unit 2), the union (unit 3), wave 2 (unit 4).
Goals
--program <tsconfig-relpath>(repeatable) restricts the run to the named program(s)--list-programsenumerates the shards, so an orchestrator can discover themby a deeper unselected tsconfig must be excluded, never reassigned to a selected ancestor
Projects and the symbol-table file set--inputand--app-nameunchanged, socan://ids are byte-identical to a whole-repo runCaveats and known risks
The reassignment bug is the whole risk.
ownerProgramfalls back to the root program for anyfile no scope contains. Filtering the spec list before assignment would silently pull files owned
by deeper tsconfigs into a selected ancestor, compiling them under the wrong config — wrong module
resolution, wrong
pathsaliases, wrong lib. Ownership must be computed globally and filteredafter. This needs a direct test, not just an end-to-end one.
Measured shard distribution on vscode (13,370 files, 96 non-empty programs):
srcextensions/copilotSo this does not deliver a low peak: the largest shard is the same tree that already measures
28.75 GB standalone at
-a 4. What it delivers is that the peak is bounded by ONE program insteadof all 96 — which is the difference between completing and being killed. There is no headroom
beyond that, and
srccannot be split further without breaking id stability.Definition of done
bun testgreen;--input/--app-nameproduce ids identical to the unsharded runsrcshard, reported against the 28.75 GB standalone baseline