Skip to content

feat(cambricon): add topk softmax provider - #958

Draft
baominghelly wants to merge 1 commit into
masterfrom
feat/cambricon-topk-softmax
Draft

feat(cambricon): add topk softmax provider#958
baominghelly wants to merge 1 commit into
masterfrom
feat/cambricon-topk-softmax

Conversation

@baominghelly

Copy link
Copy Markdown
Contributor

Summary

  • Add a native Cambricon slot 0 provider for topk_softmax in src/native/cambricon/ops/topk_softmax/.
  • Support float16, bfloat16, and float32 gating inputs with int32, uint32, or int64 expert indices.
  • Extend the existing native tests to run on Cambricon while preserving the explicit NVIDIA linked slot 16 tests.

Motivation

Cambricon currently has no native topk_softmax provider for MoE routing. This change adds the slot 0 implementation so gating probabilities, expert selection, optional bias, padding sentinels, and optional renormalization can execute on MLU.

Type of Change

  • feat — new feature / new operator / new platform
  • fix — bug fix
  • perf — performance improvement (no behavioral change)
  • refactor — code restructuring without behavior change
  • test — adding or fixing tests only
  • docs — documentation only
  • build / ci — build system or CI configuration
  • chore — tooling, formatting, or other non-code changes
  • Breaking change (requires a ! in the Conventional Commits prefix or a BREAKING CHANGE: footer)

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

Targeted Cambricon operator build and test were run. The repository smoke suite was not run, so this PR remains Draft.

cmake -S . -B /workspace/build/retest-v3-topk_softmax \
  -DAUTO_DETECT_DEVICES=OFF \
  -DWITH_CPU=ON \
  -DWITH_CAMBRICON=ON \
  -DWITH_TORCH=OFF \
  -DWITH_LINKED=OFF \
  -DGENERATE_PYTHON_BINDINGS=ON \
  -DINFINI_OPS_OPS=topk_softmax \
  -DINFINI_RT_ROOT=/workspace/install/infinirt-master-test \
  -DPython_EXECUTABLE=/torch/venv3/pytorch/bin/python3.10 \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=/workspace/build/retest-v3-topk_softmax-site/infini
cmake --build /workspace/build/retest-v3-topk_softmax --parallel "$(nproc)"
cmake --install /workspace/build/retest-v3-topk_softmax
PYTHONNOUSERSITE=1 \
PYTHONPATH=/workspace/build/retest-v3-topk_softmax-site \
LD_LIBRARY_PATH=/workspace/build/retest-v3-topk_softmax-site/infini:/workspace/install/infinirt-master-test/lib:/usr/local/neuware/lib64:/usr/local/neuware/lib \
/torch/venv3/pytorch/bin/python3.10 -m pytest -q \
  tests/test_topk_softmax.py --devices cambricon --maxfail=1

[100%] Built target ops
32 passed, 51 skipped in 0.45s

Test Results on Supported Platforms

Platform Affected Build / Smoke Result Full Result / Notes
NVIDIA No N/A - not affected Existing linked slot 16 tests remain unchanged
Iluvatar No N/A - not affected N/A - not affected
MetaX No N/A - not affected N/A - not affected
Cambricon Yes Targeted build passed tests/test_topk_softmax.py: 32 passed, 51 skipped
Moore No N/A - not affected N/A - not affected
Ascend No N/A - not affected N/A - not affected
Full `pytest` output (optional)
........ssss........ssss........ssss......s..sssssssssssssssssssssssssss [ 86%]
sssssssssss                                                              [100%]
32 passed, 51 skipped in 0.45s

Benchmark / Performance Impact

N/A - no performance benchmark was run.

Notes for Reviewers

  • Native tests continue to use the default slot 0. NVIDIA linked coverage continues to explicitly select slot 16.
  • Selection uses optional bias, but returned weights are gathered from the original softmax probabilities. Ties select the smaller expert index.
  • Padding writes the all-ones sentinel for index outputs. Cambricon uint32 cases are skipped because torch_mlu cannot construct the required uint32 test tensors.
  • Per-task scratch storage must fit in NRAM; the launcher checks the required size before launch.
  • Validation is limited to the targeted Cambricon build and operator test file; the repository smoke suite and full suite were not run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant