[https://nvbugs/6902353][chore] Waive test_transfer_lifecycle_gpu[destination_fatal]

[lowsfer]

Dev Engineer Review

• The change adds a waiver for destination_fatal under NVBUG 6902353. It does not change test code or runtime behavior.
• The waiver joins six existing waivers for the same test module. The change further reduces coverage of transfer lifecycle behavior while the underlying startup-deadline issue remains unresolved.

QA Engineer Review

• Modified tests/integration/test_lists/waives.txt. Added a SKIP entry for test_transfer_lifecycle_gpu[destination_fatal].
• The test database still lists unittest/disaggregated/test_transfer_lifecycle_gpu.py in test-db/l0_dgx_b200.yml. The waiver list now covers all seven test IDs reported for this module.
• Coverage verdict: needs follow-up. The affected test cases are waived; verify the startup fix and restore coverage when NVBUG 6902353 is resolved.

Per-File QA Perspective

• tests/integration/test_lists/waives.txt: Adds a waiver for the destination_fatal test ID. The test database still lists the test module, while the waiver list covers its seven reported cases.

Description

Waive unittest/disaggregated/test_transfer_lifecycle_gpu.py::test_transfer_lifecycle_gpu[destination_fatal] under the existing NVBUG 6902353, alongside the six sibling cases waived in #20235.

Why: the test waits 120 s for four native MPI ranks to come up. On a fresh CI node, whichever case runs first in the module pays a one-time startup cost and exceeds that deadline (AssertionError: Timed out: four native MPI ranks). The later cases start warm.

• Before [https://nvbugs/6902353][chore] waive test_transfer_lifecycle_gpu #20235 (e.g. x86-64 Multi-GPU run gh3#2826, 2026-10-13 11:56Z): baseline ran first and failed at 124.2 s; every other case, including destination_fatal (159.9 s), passed.
• [https://nvbugs/6902353][chore] waive test_transfer_lifecycle_gpu #20235 (merged 2026-10-13 16:52Z) waived baseline, cancel_before_publication, cancel_late_aux, cancel_late_kv, source_fatal and timeout_late_aux, which left destination_fatal as the first and only case. It has failed in every run since: 16 of 16 x86-64 Multi-GPU runs from 2026-10-13 20:59Z through 2026-10-16, across 14 PRs (e.g. [None][perf] Do not block the executor on KV bounce recv-region backp… #19940, [None][feat] Stream native KV events and expose routing load #19440, [trtllm-agent][https://nvbugs/6812350][fix] Raise remote MPI worker exceptions from submit_sync #19561, [None][feat] Publish serving configuration schemas #19906, [None][fix] Enable VMM CUDA IPC in Kubernetes GPU CI #20042, [None][chore] Remove ArrayView and use std::span instead #20205, [None][test] Migrate ordinary disaggregated coverage to KV cache manager V2 #20050, [None][fix] Merge back #19593 from release/1.3: Make MoE communication timeouts follow the warmup phase #20246, [None][feat] Add the PrimTS native sparse MLA FMHA for DeepSeek-V4 generation #20209, [None][perf] One-model speculative decoding: host work between steps, worker-produced target logits, KDA drafts' records #19989, [TRTLLM-16992][perf] Use paged attention for MiniMax-M3 dense layers #20159, [None][perf] primsTS dense bf16, qk-bf16/pv-fp8, fp8 fmha context bringup #20006, [None][perf] Fill the KVCM2 host tier progressively #19650), with no passes. Before the [https://nvbugs/6902353][chore] waive test_transfer_lifecycle_gpu #20235 waive it passed 6 of 6 times.

This currently fails DGX_B200-4_GPUs-PyTorch-3 for every PR whose pipeline runs the x86-64 Multi-GPU leg. Single-GPU-only PRs never run it.

Note on #20439 (unwaiving the six cases): with all seven cases enabled, the first case (baseline) is expected to hit the same cold-start timeout again, as it did before #20235. The underlying fix is to give the first launch a longer startup deadline or warm the workers up once per module, which belongs to NVBUG 6902353.

Test Coverage

Waive-only change; no code changes. DGX_B200-4_GPUs-PyTorch-3 should no longer run any test_transfer_lifecycle_gpu case.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

🤖 Generated with Claude Code

[lowsfer]

/bot run --disable-fail-fast

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA/TensorRT-LLM/.coderabbit.yaml
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 1afd2872-5492-41f6-82c5-29dd2af665ad

📥 Commits

Reviewing files that changed from the base of the PR and between f952e41 and 6d6998f.

📒 Files selected for processing: 1

• tests/integration/test_lists/waives.txt

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Walkthrough

The change adds a skip-list entry for test_transfer_lifecycle_gpu[destination_fatal] and links it to NVBUG 6902353.

Changes

GPU transfer test waiver

Layer / File(s)	Summary
Add destination_fatal waiver tests/integration/test_lists/waives.txt	The skip list now includes test_transfer_lifecycle_gpu[destination_fatal], linked to NVBUG 6902353.

Priority: ➖ Normal

Estimated code review effort: 1 (Trivial) | ~3 minutes

Change: Other

Suggested reviewers: bowenfu

Merge Risk: ⚪ Minimal · up to 6d699

This waiver removes the final runnable lifecycle case alongside six sibling cases, reducing automated coverage while the reported startup failure is tracked. The broader reported failure scope supports leaving the waiver unscoped; no separate merge-blocking risk remains.

Pre-merge checks: 5

Check name	Status	Explanation
Title check		The title clearly identifies NVBUG 6902353, the chore type, and the specific test case being waived.
Description check		The description explains the issue, rationale, affected test and CI job, expected impact, test coverage, and checklist status. It follows the required template.
Docstring Coverage		No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[tensorrt-cicd]

PR_Github #78360 [ run ] triggered by Bot. Commit: 6d6998f Link to invocation

[lowsfer]

/bot run --disable-fail-fast

[github-actions]

Automatically added "ci: full pre-merge approved" because this PR has satisfied the required GitHub review approvals. Unresolved review conversations and other required checks remain independent merge requirements.

[tensorrt-cicd]

PR_Github #78399 [ run ] triggered by Bot. Commit: 6d6998f Link to invocation

[tensorrt-cicd]

PR_Github #78360 [ run ] completed with state ABORTED. Commit: 6d6998f

Link to invocation

[lowsfer]

/bot skip --comment "waive only"

[tensorrt-cicd]

PR_Github #78419 [ skip ] triggered by Bot. Commit: 6d6998f Link to invocation

[tensorrt-cicd]

PR_Github #78399 [ run ] completed with state ABORTED. Commit: 6d6998f

Link to invocation

[tensorrt-cicd]

PR_Github/20457-6d6998f #78399 was force-killed by a newer pipeline run.
L0 job information not available (job may not have been triggered yet).

Link to superseding invocation

[tensorrt-cicd]

PR_Github #78419 [ skip ] completed with state SUCCESS. Commit: 6d6998f
Skipping testing for commit 6d6998f

Link to invocation