[PyTorch] DeepSeekV3Layer: full MoE transformer layer (MLA + DeepSeek MoE) - #3471

Open

pggPL wants to merge 59 commits into

NVIDIA:mainfrom

pggPL:deepseek_v3_layer

Commits on Aug 18, 2026

[PyTorch] Add DeepSeekV3Layer skeleton (MLA + MoE)

pggPL
and
claude
committed
45ed767
Move DeepSeekV3 skeleton to models/deepseek_v3 subpackage

pggPL
and
claude
committed
c306c6f
Add DeepSeekV3 layer entries to PyTorch API docs

pggPL
and
claude
committed
f73d04e
[PyTorch] Implement DeepSeekV3Layer: MLA + MoE from TE building blocks

pggPL
and
claude
committed
09f28a9
Add distributed EP test for DeepSeekV3 MoE/layer

pggPL
and
claude
committed
e23100b
Fix EP wgrad test collective + zero EP recv/grad buffers

pggPL
and
claude
committed
4c6e1e8
Use fused MLA RoPE kernels in MultiLatentAttention

pggPL
and
claude
committed
aa17c37

Commits on Aug 21, 2026

Add HF transformers numeric reference test for DeepSeekV3Layer

pggPL
and
claude
committed
c713af7
Docstring cleanups for lint and docs build

pggPL
and
claude
committed
88028a9
Move model-specific layers to a dedicated docs page

pggPL
and
claude
committed
5f68c9b
Drop HF-transformers comparison test from the repo

pggPL
and
claude
committed
9db495a
Docs: reduce models page to a plain API listing

pggPL
and
claude
committed
4883b17

Commits on Sep 3, 2026

Commits on Sep 8, 2026

Commits on Sep 9, 2026

[PyTorch] DeepSeekV3MoE EP: drop full-buffer zeroing of recv/grad buffers

pggPL
and
claude
committed
93a40df
[PyTorch] Add DeepSeekV3Layer expert-parallel example

pggPL
and
claude
committed
5e36ef1
[PyTorch] DeepSeekV3Layer example: naive PyTorch MoE and dense baselines

pggPL
and
claude
committed
3612879
[PyTorch] DeepSeekV3Layer example: two-level plain PyTorch MoE baseline

pggPL
and
claude
committed
eb36d7d
[PyTorch] DeepSeekV3: MLA TP gradient reduction, RoPE guards, per-forward EP buffer

pggPL
and
claude
committed
7a1693a
[PyTorch] DeepSeekV3Layer example: argument validation, gradient checks, differentiable naive probs

pggPL
and
claude
committed
49ecaff
[PyTorch] Drop example-dependent tests from test_models.py

pggPL
and
claude
committed
a131eb3
[Docs] Organize DeepSeekV3 example README around results
pggPL
committed
b5605bc
[Docs] Keep naive MoE comparison in headline results
pggPL
committed
e9fac4f
[PyTorch] DeepSeekV3Layer example README: remeasure all variants on the current code

pggPL
and
claude
committed
00bcbd9
[PyTorch] DeepSeekV3Layer example README: split NCCL EP permute from rank-skew wait in the breakdown

pggPL
and
claude
committed
4ffa639
[PyTorch] DeepSeekV3MoE: optional caller-owned EpBuffer

pggPL
and
claude
committed
0367781
[PyTorch] DeepSeekV3Layer example README: medians of three runs, breakdown from an unskewed node

pggPL
and
claude
committed
da81d1d

Commits on Sep 10, 2026

[PyTorch] Validate MoE grouping and check synchronized expert bias updates
pggPL
committed
2f7859a

Commits on Sep 17, 2026

Commits on Sep 28, 2026

Merge upstream/main into deepseek_v3_layer
pggPL
committed
793b46b

Commits on Sep 30, 2026

Commits on Oct 1, 2026

Remove benchmark commit provenance from DeepSeek README
pggPL
committed
b32a4f8
Merge upstream main for MoE Sequential ops
pggPL
committed
85612d4
Compose DeepSeek expert parallel path with MoE ops
pggPL
committed
0343fab

Commits on Oct 5, 2026

Let MoE EP ops allocate their data buffers
pggPL
committed
72abef7