Skip to content

fix(dataflow): extract nested programs in their compiler context - #127

Closed
MikeeI wants to merge 1 commit into
codellm-devkit:mainfrom
MikeeI:fix/issue-009
Closed

fix(dataflow): extract nested programs in their compiler context#127
MikeeI wants to merge 1 commit into
codellm-devkit:mainfrom
MikeeI:fix/issue-009

Conversation

@MikeeI

@MikeeI MikeeI commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Extract each program's callables with its owning compiler context.
Resolve the missing L3/L4 coverage reported in #111 while reusing one worker pool.

Evidence

Changes

  • Sequence per-program semantic and extraction work while retaining their intended overlap.
  • Merge callable graph data before whole-application summary composition.

Risks and boundaries

Verification

  • bun test test/multi-tsconfig.test.ts
  • bun test
  • bun run typecheck

I checked the relevant issues, comments, pull requests, and discussions; this pull request is not a duplicate.

Disclosure

Investigated thoroughly with GPT-5.6 at extra-high reasoning effort.
I used Oh My Pi as the agent framework.
This report is not generic or unreviewed AI-generated output.
Its claims were checked against the cited evidence.
It includes the relevant detail intended to help maintainers resolve the issue.
If reports like this are not useful to the project, please let me know.
I will refrain from submitting similar reports.
My intent is to help without wasting maintainer time or energy or discouraging their work.
Thank you for your work.

@rahlk rahlk self-assigned this Sep 2, 2026
@rahlk

rahlk commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Thanks for this, and sorry it crossed with my own fix — #137 landed on main a few hours before I saw this, solving the same issue with a different orchestration. Rather than close it unread I rebased yours onto current main (taking your core.ts / dataflow/index.ts in place of mine, since a merge of both would test neither) and ran the two head to head on real source.

Two designs. Mine extracts once after the whole call-graph loop, with a per-program AST index and disposal as it goes. Yours interleaves — each program extracted immediately after its own call-graph pass, reusing one worker pool. I genuinely expected yours to bound residency better.

Apache Superset frontend (39 tsconfigs, 1,841 modules) — identical results:

callables with CFG param_in peak
main before either fix 8,138 (73.4%) 6,475 10.3 GB
#137 (merged) 11,023 (99.4%) 17,920 10.6 GB
this PR, rebased 11,023 (99.4%) 17,920 10.5 GB

Two independent implementations landing on the same numbers is the best corroboration either of us could ask for.

microsoft/vscode at -a 4 (92 programs, no root tsconfig, 174,767 callables) — where I hoped yours would win:

peak wall outcome
#137 30.8 GB 17m47s killed (OOM)
this PR, rebased 30.55 GB 15m37s killed (OOM)

Yours is slightly leaner and about two minutes quicker, but both hit the same wall. My hypothesis that interleaving would clear it was wrong — the residual cost is not the orchestration, so I am closing this as superseded rather than swapping the merged implementation for it.

On the diff itself, since you are a new contributor here and I reviewed it properly: analyzer source and tests only, no workflow, packaging or dependency changes, no network calls, no process spawning, no dynamic evaluation. Clean.

One thing worth passing on — the commits are authored as Bonobo <github@in9.at> while the PR comes from MikeeI. Almost certainly just local git config, but you may want them to line up.

The remaining vscode gap is tracked in #112: incremental reuse and per-program sharding, which change the growth curve rather than shaving the constant. If that interests you, the interleaving you have built here is a reasonable foundation for the sharding step — it already establishes the per-program boundary that work needs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants