[Common] Remove CuTeDSL first launch protection

[kainzhong]

Description

nvidia-cutlass-dsl 4.8.0 has fixed the deadlock bug NVIDIA/cutlass#3706 so we don't need this temporary fix now.

However, since we current don't enable CuTeDSL by default and we don't specify the minimum version for CuTeDSL dependencies, users who build TE using NVTE_WITH_CUTEDSL need to install nvidia-cutlass-dsl>=4.8.0 on their own to make it work correctly.

When we enable CuTeDSL in TE by default we'll require nvidia-cutlass-dsl >= 4.8.0 in pyproject.toml and setup.py so it will become an explicit dependency requirement

Fixes # (issue)

Type of change

• Documentation change (change only to the documentation, either a fix or a new content)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Infra/Build change
• Code refactoring

Changes

Please list the changes introduced in this PR:

• Remove the first time CuTeDSL function launch protection

Checklist:

• I have read and followed the contributing guidelines
• The functionality is complete
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[greptile-apps]

Confidence Score: 5/5

[High risk] Removes serialization wrapper around GPU kernel first launch.

The PR appears safe to merge based on this review.

Findings

1.  Older CuTeDSL versions can deadlock ▶

Summary

The PR removes CuTeDSL first-launch serialization and stores TVM-FFI function handles directly in the MXFP8 kernel cache.

• No changes were made since the previous review.
• The previous deadlock concern was conceded in its thread given that CuTeDSL is currently opt-in.

_{Reviews (2) · Last reviewed commit: "remove first launch protection"}

[greptile-apps]

Older CuTeDSL versions can deadlock

If the CuTeDSL backend is enabled with nvidia-cutlass-dsl older than 4.8.0, concurrent first MXFP8 launches on multiple GPUs now call the TVM-FFI function without the protection that prevented a deadlock. Transformer Engine does not require 4.8.0 or reject older installations, so those users can encounter the deadlock again. Require the fixed version before removing the guard, or keep the guard for older versions.

[kainzhong]

When NVTE_WITH_CUTEDSL becomes on by default we will require nvidia-cutlass-dsl >= 4.8.0. Currently it's just in but not enabled by default (and as you can see the current codebase doesn't require nvidia-cutlass-dsl at all

[greptile-apps]

That addresses the default-installation path: users who do not explicitly enable NVTE_WITH_CUTEDSL will not hit this code, and the dependency can be raised when the backend becomes default.

The remaining risk is limited to users who opt in today while still having nvidia-cutlass-dsl < 4.8.0; for those users, removing the guard can reintroduce the documented first-launch deadlock. If supporting that opt-in combination is intentional, the guard should remain until the dependency requirement is enforced. Otherwise, this is reasonable as a forward-looking migration assumption.

[greptile-apps]

That’s fair. Since NVTE_WITH_CUTEDSL is currently opt-in and Transformer Engine does not declare or enforce a specific nvidia-cutlass-dsl dependency yet, the deadlock concern only affects users who manually enable CuTeDSL with an older DSL version. Once the backend is enabled by default, enforcing nvidia-cutlass-dsl >= 4.8.0 will be necessary; for the current opt-in state, this change is reasonable and I’m resolving the comment.

[ptrendx]

I think we need to include some note in the release notes (@kainzhong could you make some note in the PR description along those lines so that the automatic release note draft picks it up hopefully) that for this functionality we require 4.8.0+?

[ptrendx]

/te-ci

[kainzhong]

I think we need to include some note in the release notes (@kainzhong could you make some note in the PR description along those lines so that the automatic release note draft picks it up hopefully) that for this functionality we require 4.8.0+?

@ptrendx I changed some of my description though I don't know how the automatic release note works. Is the current description correct?