Skip to content

feat(cli): --program selector, so one program can be analysed at a time #146

Description

@rahlk

Problem

Whole-repository analysis holds every program's ts-morph Project at once. On vscode that is 96
programs and it is what kills -a 4 (28.0-31.7 GB, exit 133). There is no way to analyse one
program at a time, which is the precondition for
the sharded, two-wave design (#112 Step 4).

Scope boundary

Unit 1 of the spec: the program selector only. Emits a normal, complete analysis.json for the
selected program(s) — it does not union shards (unit 3) or stitch cross-shard dataflow (unit 4).

Not in scope: IR persistence (unit 2), the union (unit 3), wave 2 (unit 4).

Goals

  • --program <tsconfig-relpath> (repeatable) restricts the run to the named program(s)
  • --list-programs enumerates the shards, so an orchestrator can discover them
  • Ownership is still computed against all discovered programs, then filtered — a file owned
    by a deeper unselected tsconfig must be excluded, never reassigned to a selected ancestor
  • Restricts both the constructed Projects and the symbol-table file set
  • --input and --app-name unchanged, so can:// ids are byte-identical to a whole-repo run

Caveats and known risks

The reassignment bug is the whole risk. ownerProgram falls back to the root program for any
file no scope contains. Filtering the spec list before assignment would silently pull files owned
by deeper tsconfigs into a selected ancestor, compiling them under the wrong config — wrong module
resolution, wrong paths aliases, wrong lib. Ownership must be computed globally and filtered
after. This needs a direct test, not just an end-to-end one.

Measured shard distribution on vscode (13,370 files, 96 non-empty programs):

shard files share
src 8,966 67.1%
extensions/copilot 2,659 19.9%
remaining 94 <= 226 each ~13%

So this does not deliver a low peak: the largest shard is the same tree that already measures
28.75 GB standalone at -a 4. What it delivers is that the peak is bounded by ONE program instead
of all 96 — which is the difference between completing and being killed. There is no headroom
beyond that, and src cannot be split further without breaking id stability.

Definition of done

  • Selecting every program reproduces the whole-repo run's module set exactly
  • A file owned by a deeper unselected tsconfig is absent from the output, not reassigned
  • bun test green; --input/--app-name produce ids identical to the unsharded run
  • Peak RSS measured on vscode's src shard, reported against the 28.75 GB standalone baseline

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions