Add DeepSeek-V4-Pro FP4 B200 Dynamo TensorRT-LLM recipes / 添加 DeepSeek-V4-Pro FP4 B200 Dynamo TensorRT-LLM 配置 - #2552
Add DeepSeek-V4-Pro FP4 B200 Dynamo TensorRT-LLM recipes / 添加 DeepSeek-V4-Pro FP4 B200 Dynamo TensorRT-LLM 配置#2552RohitNagraj wants to merge 2 commits into
Conversation
添加 DSV4 FP4 B200 Dynamo TensorRT-LLM 配置及配套配方。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
更新变更日志中的拉取请求链接。
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| echo "Cloning srt-slurm repository..." | ||
| SRT_REPO_DIR="srt-slurm" | ||
| rm -rf "$SRT_REPO_DIR" | ||
| if [[ $MODEL_PREFIX == "dsv4" ]]; then | ||
| if [[ $MODEL_PREFIX == "dsv4" && $FRAMEWORK == "dynamo-trt" ]]; then | ||
| mkdir -p "$SRT_REPO_DIR" | ||
| git -C "$SRT_REPO_DIR" init --quiet || exit 1 | ||
| git -C "$SRT_REPO_DIR" fetch --quiet --depth 1 \ | ||
| https://github.com/NVIDIA/srt-slurm.git refs/pull/274/head || exit 1 | ||
| git -C "$SRT_REPO_DIR" checkout --quiet 22132f7449769315a441b1061dbe1d67435887bb || exit 1 | ||
| cd "$SRT_REPO_DIR" || exit 1 | ||
| mkdir -p recipes/trtllm/deepseek-v4 | ||
| cp -rT "$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/trtllm/deepseek-v4" \ | ||
| recipes/trtllm/deepseek-v4 || exit 1 | ||
| cp "$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/configs/moe_load_balancer_ctx_ep4_384.yaml" \ | ||
| "$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/configs/moe_load_balancer_gen_ep8_slots384.yaml" \ | ||
| configs/ || exit 1 | ||
| sed -i 's/CONTAINER_REMAP_ROOT_EXPORT = {"ENROOT_REMAP_ROOT": "yes"}/CONTAINER_REMAP_ROOT_EXPORT = {"ENROOT_REMAP_ROOT": "no"}/' \ | ||
| src/srtctl/core/slurm.py | ||
| elif [[ $MODEL_PREFIX == "dsv4" ]]; then | ||
| git clone https://github.com/NVIDIA/srt-slurm.git "$SRT_REPO_DIR" || exit 1 | ||
| cd "$SRT_REPO_DIR" || exit 1 | ||
| git checkout aflowers/vllm-gb200-v0.20.0 || exit 1 |
There was a problem hiding this comment.
🟡 The sed -i patch that flips ENROOT_REMAP_ROOT in src/srtctl/core/slurm.py has no || exit 1 or match check, unlike every other step in this same block (git init/fetch/checkout, cp -rT, cp configs), which all guard with || exit 1. sed -i exits 0 even on a no-match, so if the pinned commit's file content ever drifts from this exact string, the substitution silently no-ops and the job reverts to upstream'''s ENROOT_REMAP_ROOT=yes default instead of failing loudly. Consider adding a grep -q check on the target string (or git diff --quiet after) to fail fast like the surrounding lines.
Extended reasoning...
What the bug is: In runners/launch_b200-nscale-slurm.sh (lines 65-86), the newly added dsv4+dynamo-trt branch clones a pinned commit of srt-slurm, copies in recipe/config files, and then patches src/srtctl/core/slurm.py with sed -i 's/CONTAINER_REMAP_ROOT_EXPORT = {"ENROOT_REMAP_ROOT": "yes"}/.../"no".../' to flip the enroot root-remap default for this run. Every other step in the same block — git init, git fetch, git checkout, cd, cp -rT, and the two cp calls for the load-balancer configs — is guarded with || exit 1. The final sed -i line is the only step with no such guard and no verification that the substitution actually happened.
Why existing code does not prevent it: GNU sed -i returns exit status 0 even when the given pattern matches zero lines in the file — it only fails on things like a missing file or invalid syntax, not a non-matching substitution. So there is no shell-level signal that would trip set -e/|| exit 1 semantics if the pattern simply fails to match.
How it manifests / concrete walkthrough:
- The script pins
srt-slurmto commit22132f7449769315a441b1061dbe1d67435887bband checks it out with|| exit 1, so today the exact stringCONTAINER_REMAP_ROOT_EXPORT = {"ENROOT_REMAP_ROOT": "yes"}is present insrc/srtctl/core/slurm.pyand the substitution succeeds silently. - At some future point a maintainer bumps that pinned SHA (as already happens routinely for other pins in this file, e.g. the
vllmandkimi-k2.6branches checkout different refs over time) because upstream reformats the line, renames the constant, or restructures the dict literal. - The
sed -icommand still runs and still exits 0, but matches nothing —src/srtctl/core/slurm.pyis left completely unchanged, still containing"ENROOT_REMAP_ROOT": "yes". - The script continues past this line exactly as if the patch had applied. The subsequent
srtctl applyjob launches with upstream's default enroot root-remap behavior instead of the intended override. - Whatever behavior difference
ENROOT_REMAP_ROOT=nowas meant to produce (avoiding unwanted root-mapping in the container) is silently absent. If this surfaces as a job failure downstream (e.g. a permissions issue inside the container), a maintainer debugging it has to work backward through the enroot/container stack rather than seeing a clean "patch didn't apply" script exit at the point of the actual problem.
Impact: This is a robustness/quality gap, not a bug that fires with the code as committed today (the pinned SHA's content is known and matches). It only manifests when the pin is bumped later without also updating this sed pattern — but when that happens, the failure mode is a silent behavioral regression that is much harder to diagnose than a clear script exit at the point of divergence.
Suggested fix: Add a guard consistent with the rest of the block, e.g. grep -q '\''CONTAINER_REMAP_ROOT_EXPORT = {"ENROOT_REMAP_ROOT": "yes"}'\'' src/srtctl/core/slurm.py || exit 1 before the sed -i, or verify the file actually changed afterward (e.g. via git diff --quiet src/srtctl/core/slurm.py && exit 1) so a future SHA bump that breaks this patch fails the script immediately instead of degrading into an untraceable runtime issue.
| description: | ||
| - "Add the DeepSeek-V4-Pro FP4 B200 Dynamo TensorRT-LLM disaggregated configuration with STP and MTP recipe variants." | ||
| - "Run on b200-new with the checkpoint at /scratch/models/DeepSeek-V4-Pro and batch_1/benchmark Slurm settings." | ||
| - "Stage the TensorRT-LLM recipes and load-balancer configs through launch_b200-nscale-slurm.sh." |
There was a problem hiding this comment.
🟡 The new perf-changelog.yaml entry for dsv4-fp4-b200-dynamo-trt ends with pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX — a literal unfilled placeholder instead of this PR's actual number (2552). This breaks the traceability the field exists for; please replace XXX with 2552 before merging.
Extended reasoning...
The new perf-changelog.yaml entry added for the dsv4-fp4-b200-dynamo-trt config key ends with:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXXXX is a literal, unfilled template placeholder — not a real PR number. This is the last line of the diff hunk that adds the new changelog entry, so it's easy to miss in review since it comes after the more substantive config-keys/description fields.
Why this happens / why nothing catches it today: the changelog entry is presumably drafted before the PR number is known (since GitHub assigns the PR number on creation, and the entry has to be written into the branch beforehand), and there's no lint/CI check that validates pr-link values against a pull/\d+ pattern or cross-checks them against the actual PR number. Every other entry in the file (e.g. the immediately preceding ones referencing PRs 2528, 2520, 2319) has a real numeric link, confirming this is simply a forgotten find-and-replace rather than an intentional convention.
Impact: pr-link exists specifically so a future reader can trace a benchmark/perf change back to the PR that introduced it. A literal XXX breaks that traceability — anyone auditing perf-changelog.yaml for the dsv4-fp4-b200-dynamo-trt entry hits a dead/placeholder link instead of the originating PR (#2552 for this PR).
Proof (step-by-step):
- Open
perf-changelog.yamlat the tail of the diff. - Find the new entry with
config-keys: - dsv4-fp4-b200-dynamo-trt. - Read its final line:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX. - Compare against the PR metadata: this PR is Add DeepSeek-V4-Pro FP4 B200 Dynamo TensorRT-LLM recipes / 添加 DeepSeek-V4-Pro FP4 B200 Dynamo TensorRT-LLM 配置 #2552.
XXX!=2552andXXXis not a valid PR number at all — confirming the placeholder was never filled in.
Fix: replace XXX with 2552 in that line:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2552This is a documentation/traceability defect only — it doesn't affect the benchmark sweep, the recipe configs, or any runtime behavior, so it's a nit rather than a blocking issue, but it should still be fixed before merge since it's a one-line change with a clear correct value.
| use_chat_template: true | ||
| custom_tokenizer: "sa_bench_tokenizers.sglang_deepseek_v4.SGLangDeepseekV4Tokenizer" | ||
| frontend: | ||
| type: dynamo |
There was a problem hiding this comment.
🟡 In disagg-b200-8k1k-1p5d-dep8-tep8-b32-mtp0.yaml, decode is non-DP (tep8) with 5 decode workers and max_batch_size: 32, giving a decode capacity of 5*32=160, but benchmark.concurrencies is only "40" — the exact value from the sibling b8 recipe, left over from before max_batch_size was bumped from 8 to 32. Every other non-DP tep8 recipe in this PR sets concurrencies = decode_workers * max_batch_size exactly, so this should be "160" (and the matching conc-list: [40] in configs/nvidia-master.yaml needs the same fix).
Extended reasoning...
The bug: disagg-b200-8k1k-1p5d-dep8-tep8-b32-mtp0.yaml configures decode as non-data-parallel (enable_attention_dp: false, i.e. tep8) with resources.decode_workers: 5 and trtllm_config.decode.max_batch_size: 32. Total decode batch capacity is therefore 5 workers × 32 = 160 concurrent requests. However benchmark.concurrencies is set to "40", and the identical [40] conc-list appears for this recipe in configs/nvidia-master.yaml.
How this was introduced: This recipe's sibling, disagg-b200-8k1k-1p5d-dep8-tep8-b8-mtp3.yaml, has the same decode_workers: 5 but max_batch_size: 8, and its concurrency is correctly "40" (5×8). The b32-mtp0 recipe's concurrency value of 40 is an exact match to that sibling's value — strong evidence this recipe was cloned from the b8 config and max_batch_size/max_num_tokens/cuda_graph_config.batch_sizes were bumped to 32 without updating benchmark.concurrencies to match.
Why the existing pattern doesn't catch it: Every other non-DP tep8 recipe added in this PR follows concurrencies = decode_workers * max_batch_size exactly: 1p4d-tep8-b4-mtp0 (4×4=16), 1p4d-tep8-b4-mtp3 (4×4=16), and the b8 sibling itself (5×8=40). The DP (enable_attention_dp: true) recipes follow a separate, also-consistent formula (workers × max_batch_size × 8), which independently confirms the tep8 formula is a real invariant in this recipe family rather than coincidence — this one recipe is the single outlier out of 16.
Impact: disagg-b200-8k1k-1p5d-dep8-tep8-b32-mtp0.yaml's decode config declares cuda_graph_config.batch_sizes up to 32 and max_batch_size: 32, clearly intended to exercise batch-32 decode. With concurrency capped at 40 spread across 5 tep8 workers (~8 requests/worker), the benchmark never drives decode utilization anywhere close to its configured capacity — the 16/24/32 CUDA graph sizes never get exercised. The sweep still completes and produces a data point, but that point under-represents the recipe's actual achievable throughput/latency, which matters in a repo whose primary output is these benchmark numbers, and wastes the 6-node B200 allocation this recipe consumes.
Step-by-step proof:
- Recipe declares
resources.decode_workers: 5,gpus_per_decode: 8,trtllm_config.decode.enable_attention_dp: false(tep8, non-DP). trtllm_config.decode.max_batch_size: 32— each of the 5 decode workers can hold up to 32 in-flight requests.- Expected total decode capacity = 5 × 32 = 160, matching the pattern from every other tep8 sibling in the diff (e.g. b8-mtp3: 5×8=40, which is correctly set to 40).
- Actual
benchmark.concurrencies="40"— 4x below the configured capacity, and identical to the b8 sibling's correct value of 40. configs/nvidia-master.yaml'sdsv4-fp4-b200-dynamo-trtentry for this recipe also listsconc-list: [40], showing the error is a single root-cause mistake (concurrency not scaled with max_batch_size) reflected consistently in both files, not a one-off transcription slip.
Fix: Update benchmark.concurrencies to "160" in the yaml recipe and conc-list: [40] to conc-list: [160] for this recipe's entry in configs/nvidia-master.yaml.
| echo "Cloning srt-slurm repository..." | ||
| SRT_REPO_DIR="srt-slurm" | ||
| rm -rf "$SRT_REPO_DIR" | ||
| if [[ $MODEL_PREFIX == "dsv4" ]]; then | ||
| if [[ $MODEL_PREFIX == "dsv4" && $FRAMEWORK == "dynamo-trt" ]]; then | ||
| mkdir -p "$SRT_REPO_DIR" | ||
| git -C "$SRT_REPO_DIR" init --quiet || exit 1 | ||
| git -C "$SRT_REPO_DIR" fetch --quiet --depth 1 \ | ||
| https://github.com/NVIDIA/srt-slurm.git refs/pull/274/head || exit 1 | ||
| git -C "$SRT_REPO_DIR" checkout --quiet 22132f7449769315a441b1061dbe1d67435887bb || exit 1 | ||
| cd "$SRT_REPO_DIR" || exit 1 | ||
| mkdir -p recipes/trtllm/deepseek-v4 | ||
| cp -rT "$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/trtllm/deepseek-v4" \ | ||
| recipes/trtllm/deepseek-v4 || exit 1 | ||
| cp "$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/configs/moe_load_balancer_ctx_ep4_384.yaml" \ | ||
| "$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/configs/moe_load_balancer_gen_ep8_slots384.yaml" \ | ||
| configs/ || exit 1 | ||
| sed -i 's/CONTAINER_REMAP_ROOT_EXPORT = {"ENROOT_REMAP_ROOT": "yes"}/CONTAINER_REMAP_ROOT_EXPORT = {"ENROOT_REMAP_ROOT": "no"}/' \ | ||
| src/srtctl/core/slurm.py | ||
| elif [[ $MODEL_PREFIX == "dsv4" ]]; then | ||
| git clone https://github.com/NVIDIA/srt-slurm.git "$SRT_REPO_DIR" || exit 1 | ||
| cd "$SRT_REPO_DIR" || exit 1 | ||
| git checkout aflowers/vllm-gb200-v0.20.0 || exit 1 |
There was a problem hiding this comment.
🟡 The dsv4+dynamo-trt branch does a shallow --depth 1 fetch of refs/pull/274/head (a mutable, open upstream PR ref) and then checks out a pinned SHA that only happens to be that ref's current tip. Since the repo is wiped and re-cloned on every run, if NVIDIA/srt-slurm PR #274 ever gets new commits or a force-push, the next sweep's depth-1 fetch won't contain the pinned SHA and git checkout will fail with fatal: reference is not a tree: <sha>, breaking the whole b200-nscale dsv4 dynamo-trt path until re-pinned. Fix by fetching the SHA directly (git fetch --depth 1 <url> <sha>), increasing fetch depth, or full-cloning like the sibling branches in this file.
Extended reasoning...
What the bug is. In the new dsv4+dynamo-trt branch of runners/launch_b200-nscale-slurm.sh (lines ~65-71), the script does:
git -C "$SRT_REPO_DIR" init --quiet || exit 1
git -C "$SRT_REPO_DIR" fetch --quiet --depth 1 \
https://github.com/NVIDIA/srt-slurm.git refs/pull/274/head || exit 1
git -C "$SRT_REPO_DIR" checkout --quiet 22132f7449769315a441b1061dbe1d67435887bb || exit 1A --depth 1 fetch retrieves only the single commit object currently at the tip of the given ref — no ancestor history. git checkout <sha> does not trigger any network fetch; it can only succeed if that commit object is already present in the local object store. Today this works only because the pinned SHA 22132f7... happens to equal the current tip of refs/pull/274/head.
Why this breaks in the future, not today. refs/pull/274/head is not a tag or an immutable ref — it's GitHub's live pointer to the head of an open, actively-developed upstream pull request. Any future push, rebase, or force-push to that PR moves the ref tip. Because this script does rm -rf "$SRT_REPO_DIR" and re-inits the repo from scratch on every single run (no persistent clone/cache across runs), the very next sweep after PR #274 moves will shallow-fetch the new tip commit, and the pinned SHA 22132f7... will no longer be present as a fetchable object (it's now an ancestor or a completely orphaned commit from that ref's perspective). The subsequent git checkout 22132f7... will fail with fatal: reference is not a tree: 22132f7449769315a441b1061dbe1d67435887bb, and the script has set -x with unguarded || exit 1 chains, so the whole b200-nscale dsv4 dynamo-trt sweep path dies at this step.
Why nothing else in the file catches this. The other two branches of the same if/elif/else in this file (dsv4 non-trt, and kimik2.6) both do a full, unbounded git clone before checking out a branch or fixed commit — a full clone retrieves complete history, so a pinned SHA stays reachable indefinitely regardless of where the ref's tip later moves. Only the new dynamo-trt branch uses the depth-1-fetch-then-checkout-by-SHA pattern, which structurally depends on the pinned SHA coinciding with the ref tip at fetch time.
Step-by-step proof (reproducible today):
rm -rf repo && git init repo && cd repogit fetch --depth 1 https://github.com/NVIDIA/srt-slurm.git refs/pull/274/head— succeeds, fetches only the current tip commit object.git checkout 22132f7449769315a441b1061dbe1d67435887bb— succeeds only because this SHA is that tip today.- Now simulate the PR being pushed to (a maintainer adds a commit, or force-pushes): the ref's tip SHA changes to some
<new-sha>. - Repeat step 1-2 with a clean object store (exactly what
rm -rf "$SRT_REPO_DIR"+ freshgit initproduces on the next CI run):git fetch --depth 1 ... refs/pull/274/headnow fetches only<new-sha>, not22132f7.... git checkout 22132f7449769315a441b1061dbe1d67435887bbnow fails:fatal: reference is not a tree: 22132f7449769315a441b1061dbe1d67435887bb.
Impact. Every b200-nscale dsv4 dynamo-trt benchmark job fails at container-setup time until someone notices the CI failure and re-pins to a currently-fetchable SHA (or fixes the fetch strategy). This is a real, mechanically-verified defect, but it is latent — it does not fail today, only after an external, unversioned trigger (an upstream maintainer's push to their own open PR) that this PR's author has no control over and that could happen at any point after merge.
Fix options, any of which resolves it: fetch the specific commit directly (git fetch --depth 1 https://github.com/NVIDIA/srt-slurm.git 22132f7449769315a441b1061dbe1d67435887bb, which GitHub's git server supports for reachable commits even without a named ref), increase the fetch depth enough to cover history back to the pinned SHA, or simply full-clone like the sibling dsv4/kimik2.6 branches in this same file do.
| transceiver_runtime: PYTHON | ||
| cuda_graph_config: | ||
| batch_sizes: | ||
| - 1 | ||
| - 2 | ||
| - 4 | ||
| enable_padding: true | ||
| enable_attention_dp: true | ||
| enable_lm_head_tp_in_adp: true | ||
| kv_cache_config: | ||
| dtype: fp8 | ||
| enable_block_reuse: false | ||
| free_gpu_memory_fraction: 0.8 | ||
| tokens_per_block: 128 | ||
| max_batch_size: 2 | ||
| max_num_tokens: 8 |
There was a problem hiding this comment.
🟡 In disagg-b200-8k1k-1p4d-dep8-dep8-b2-eplb384-mtp3.yaml, decode's cuda_graph_config.batch_sizes is [1, 2, 4] but decode.max_batch_size is 2, so the batch-4 CUDA graph can never be exercised. This looks like a copy-paste leftover from the sibling b4 recipe (disagg-b200-8k1k-1p2d-dep8-dep8-b4-eplb384-mtp3.yaml, which correctly uses [1, 2, 4] with max_batch_size: 4) — batch_sizes should be trimmed to [1, 2] here.
Extended reasoning...
The bug: In benchmarks/multi_node/srt-slurm-recipes/trtllm/deepseek-v4/8k1k/b200/disagg-b200-8k1k-1p4d-dep8-dep8-b2-eplb384-mtp3.yaml, the decode block sets:
max_batch_size: 2
max_num_tokens: 8
cuda_graph_config:
batch_sizes:
- 1
- 2
- 4
enable_padding: trueTensorRT-LLM's decode-side batching can never dispatch more than max_batch_size (2) requests at once, so the batch-size-4 CUDA graph entry is unreachable — no request stream can ever fill it. This is purely a config-consistency issue, not a crash: TRT-LLM either ignores/clamps cuda-graph batch sizes above max_batch_size or wastes one extra graph capture (time + a bit of GPU memory) for a size it will never use.
Why this happened: Every other recipe added in this PR keeps the largest cuda_graph_config.batch_sizes entry exactly equal to max_batch_size — e.g. the sibling disagg-b200-8k1k-1p2d-dep8-dep8-b4-eplb384-mtp3.yaml (same dep8-dep8/eplb384/mtp3 template, just b4 instead of b2) uses the identical list [1, 2, 4] but with max_batch_size: 4. The max_num_tokens: 8 in the b2 file (2 requests × 4 tokens/step under MTP draft_len=3) further corroborates that this file was derived from the b4 template by reducing max_batch_size/max_num_tokens from 4→2, but the batch_sizes list was never trimmed to match.
Step-by-step proof:
- Decode's scheduler is capped at
max_batch_size: 2— it will never admit a 3rd or 4th concurrent request. - At startup, TensorRT-LLM captures a CUDA graph for each entry in
batch_sizes: sizes 1, 2, and 4. - Because the scheduler cap is 2, no runtime batch can ever reach size 4, so the size-4 graph is captured but never invoked.
- Cross-checking against all 16 recipes in this PR (b4, b8, b16, b32, b64, b128, b256 variants), every single one keeps
max(batch_sizes) == max_batch_size; this b2 file is the sole outlier.
Fix: Change batch_sizes to [1, 2] to match max_batch_size: 2, consistent with every other recipe in this PR.
Why nit, not normal: This doesn't break the run — TensorRT-LLM handles cuda-graph batch sizes above the scheduler cap gracefully (filtering/clamping), so the worst-case impact is a modest amount of wasted graph-capture time and GPU memory at startup, not a functional failure or incorrect output. It's a legitimate cleanup that all three independent verifiers agreed should be fixed but flagged as non-blocking.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31436165344 |
1 participant
Summary / 摘要
Add
dsv4-fp4-b200-dynamo-trtwith 16 disaggregated STP and MTP recipes for B200.Run the configuration on
b200-newwith/scratch/models/DeepSeek-V4-Proandbatch_1/benchmark.Extend
launch_b200-nscale-slurm.shto stage the TensorRT-LLM recipes and load-balancer configs.添加
dsv4-fp4-b200-dynamo-trt，包含 16 个面向 B200 的解耦式 STP 和 MTP 配方。配置使用
b200-new运行器，模型路径为/scratch/models/DeepSeek-V4-Pro，Slurm 分区和账户分别为batch_1与benchmark。扩展
launch_b200-nscale-slurm.sh，用于准备 TensorRT-LLM 配方和负载均衡配置。