Skip to content

feat(moe): add a generic heterogeneous stage planner - #577

Draft
davide221 wants to merge 2 commits into
codex/ds4-overlapped-moe-balancefrom
codex/generic-heterogeneous-stage-planner
Draft

feat(moe): add a generic heterogeneous stage planner#577
davide221 wants to merge 2 commits into
codex/ds4-overlapped-moe-balancefrom
codex/generic-heterogeneous-stage-planner

Conversation

@davide221

@davide221 davide221 commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a model- and vendor-neutral aligned two-owner stage planner with explicit, whole-stage, and rate-balanced policies
  • partition common shared/dense FFN storage and execution while retaining one peer synchronization and one final join
  • optionally fuse each owner-local shared contribution into the routed-owner reduction
  • make route-quota validation derive from configured top-k and add a q=5 GPU alignment oracle
  • document the exact qualification matrix and remove the measured-slower four-output join experiment

Qualification

  • test_server_unit heterogeneous_stage: 5 passed, 0 failed
  • full test_deepseek4_unit: passed on the R9700 + Strix pair
  • final exact q=5 2K run: 88.637 tok/s median, 88.600-88.734 range
  • response SHA-256: 0f785a7ffa406498aafb14553966eaed0f52220fed0f7cc016b66921d104d194 on all measured runs

No slower experimental partition is enabled by default. The current 88.6 tok/s route-balanced profile remains the qualified behavior; the generic planner is available for model/hardware-specific qualification.

Stacked on #575.

Review in cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant