Add MiniMax-Music3 community model

[JoeMattie]

Adds minimax_music3: lyrics- and caption-conditioned song generation with MiniMaxAI/MiniMax-Music3 (44.1 kHz stereo, up to six minutes). The port follows the diffusers MiniMaxMusic3ModularPipeline reference: a Qwen3-8B autoregressive stage emits one semantic code per 40 ms frame with classifier-free guidance, a 4-layer RVQ depth decoder fills seven residual codebooks per frame, the fused per-frame hidden states condition a 36-layer flow-matching transformer over overlapping 200-frame windows, and a DAC-style Flow-VAE decoder renders stereo audio.

Full documentation with package layout, conversion commands, and options: docs/community_models/minimax_music3.md. Design notes: docs/proposals/minimax_music3.md.

Reuse and implementation notes

• The global LM uses the shared QwenCausalDecodeRuntime for prefill. Decode runs the conditional/unconditional CFG pair as one batch-2 graph built from the shared QwenDecoderLayerModule::build_with_static_cache_tail with family-owned per-branch KV caches, so the 8B weights stream once per frame; the two sequences are always position-aligned, so one KV write slot and mask serve both.
• The depth decoder runs its seven codebook steps as a single unrolled graph with on-device sampling: CFG, a top-k mask from ggml_top_k, host-supplied Gumbel noise, argmax. Gumbel-max over the masked logits draws exactly from the reference's renormalized top-k distribution, and zero noise reduces to greedy decoding for the parity probe.
• The flow transformer runs flash attention with F16 weights and activations (F32 norms and residuals). The DiT and depth decoder are stored F16 rather than the checkpoint's BF16: the CUDA BF16 matmul path is several times slower than F16 on Ampere, the weights fit F16's range with wide margin (max magnitude about 3.2), and parity is equal or better.
• The LM conversion slices the 200k-row lm_head to the 16385 sampleable rows; the vocoder conversion folds torch weight-norm pairs.

Framework changes

• qwen_decoder.cpp: the static-cache decode tail's post-attention reshape was hardcoded {1, 1, heads * dim}; it now derives batch and steps from the input shape. Identical behavior for every existing batch-1 caller; verified at runtime with NeuTTS 2E (neutts_2e_orig, which decodes through this exact path): a CUDA TTS generation after the change produces healthy audio at RTF 0.48.
• package.cpp: the multi-file-package model-spec fallback that covered minimax_h3 now also covers minimax_music3.
• New ggml-quantize-raw tool target (tools/ggml_quantize_raw.c): a stdin/stdout ggml_quantize_chunk helper for tensor types gguf-py cannot quantize (K-quants); scripts/minimax_h3/convert_dit_gguf.py already expected such a binary.
• Native WebUI: catalog and parameter entries for the family, and an entry-file rule in catalog.ts (the multi-component package loads via lm_*.gguf, like minimax_h3's dit.gguf).

Build and run

scripts/build_linux.sh --backend cuda --target audiocpp_cli --target audiocpp_server
hf download MiniMaxAI/MiniMax-Music3 --local-dir models/MiniMax-Music3-hf \
  --exclude "qwen_7B/qwen_7B/*.safetensors"
# component conversions: see docs/community_models/minimax_music3.md

build/linux-cuda-release/bin/audiocpp_cli \
  --task gen --family minimax_music3 \
  --model models/MiniMax-Music3-GGUF/lm_q8_0.gguf \
  --backend cuda --threads 8 \
  --text "$CAPTION" --request-option lyrics="$LYRICS" \
  --request-option duration_seconds=32 --seed 7 --metrics --out song.wav

Validation

Component parity against diffusers 0.40.0.dev0 (fixtures via tests/minimax_music3/reference_dump.py, C++ side via tests/minimax_music3/minimax_music3_component_probe.cpp, built with -DENGINE_BUILD_WARMBENCH=ON):

Component	Result
Tokenizer (prompt template, caption cleaning, lyrics normalization)	exact id match
LM prefill (both CFG branches)	corr 0.9998 Q8_0 / 0.993 Q4_K, argmax match
LM batched CFG decode step	corr 0.9995+ vs reference batched KV-cache forward
RVQ depth decoder (greedy rollout)	all 8 codes exact, hidden max diff 2.1e-3
Condition encoder	max diff 5.7e-5
Flow transformer forward	corr 0.99998
Vocoder	about 48 dB SNR

End-to-end generations (8 s and 32 s, multiple prompts/seeds) produce structured stereo music with intact window stitching. A demo output is published with the weights: samples/blues_32s_seed7_q8_0.wav.

Long-lived server session: three sequential /v1/tasks/run generations against one loaded model all succeeded, with idle VRAM stable between requests (451 / 467 / 467 MiB; mem_saver defaults on and frees each phase's weights after the request). Peak VRAM during a request is about 14 GB in the autoregressive phase.

python3 tools/check_loader_catalog_sync.py --self-test and the main sync check both pass; every file advertised by both spec packages resolves against the download repo (verified with per-file HTTP checks).

Performance and memory

RTX 3090, CUDA 13.3, 32 s of audio at 30 flow steps:

Configuration	AR	Flow	Vocode	Wall	RTF
Q8_0 LM (default package)	23.9 s	37.0 s	2.0 s	66.6 s	2.08
Q4_K LM	22.0 s	37.3 s	2.0 s	64.5 s	2.02

Stage timings are logged as minimax_music3.{ar_lm_decode,ar_depth,ar,flow,vocode,total}_ms. VRAM peaks around 14 GB during the autoregressive phase and around 8 GB during the flow phase; mem_saver defaults on and loads each phase's weights on demand.

Known limitations

• CUDA is the tested backend (runtime.tags: ["cuda", "gguf"]); the components also run on CPU (used by parity tests) but far below real time, and Vulkan/HIP/Metal are untested.
• Offline generation only; no streaming output.
• Sampling is distribution-exact against the reference recipe (top-k, CFG scales) but not bit-matched to torch's RNG stream, so a given seed produces different (equally valid) songs than the Python pipeline.
• The batched decode attends over the full KV capacity each step; for six-minute songs a bucketed cache view would shave some late-song cost.
• The spec's download repo currently points at joemattie/MiniMax-Music3-GGUF, which hosts the converted package in the installer's nested layout so the model manager and native UI install it today. Happy to have it mirrored into audio-cpp/audio.cpp-gguf and flip the repo id whenever you prefer.

🤖 Generated with Claude Code

[JoeMattie]

Could a maintainer tag this with the new model label? I don't have triage rights to apply it myself. Thanks!

[0xShug0]

@JoeMattie Thank you for the PR! We quietly made the model public this morning but haven’t announced it yet. https://github.com/0xShug0/audio.cpp/tree/preview/minimax-music-3 and GGUFs are at https://huggingface.co/audio-cpp/MiniMax-Music3-GGUF.

Would you like to give it a try? I think the best way is to apply your improvements or pieces we’re missing to preview/minimax-music-3 and hopefully we can address the issues mentioned here #230 (comment).

I really appreciate you taking the time to work on Music 3. And definitely don’t let this discourage you from contributing more models!

[JoeMattie]

Thanks, and no discouragement at all. Nice port! I took you up on it and applied my pieces on top of preview/minimax-music-3: #243.

Highlights from testing the preview branch on my box (RTX 3090, CUDA 13.3):

• Found and fixed a silent output-corruption bug: BF16-stored conv kernels break the CUDA conv path (the naive im2col lowering asserts, the fast path silently saturates the whole flow output into full-scale noise). Your published GGUFs dodge it because their convs happen to be F32, but the branch's converter regenerates them as BF16, and weight_type=bf16 converts them at load, so it's reachable both ways. MiniMax Music3 preview: fix BF16 conv corruption, repair converter, wire web UI #243 guards it at load and pins convs to F32 in the converter.
• Repaired the converter as shipped (variant-spec crash, spec source matching, and the missing config/ sidecars) and verified a full from-scratch package conversion.
• Added the native web UI wiring (catalog, params, entry-file rule, duration_seconds alias).

After the fixes your default q4_k config benches at RTF ~2.1 for 32 s here, and interestingly BF16 flow beats all-F16 on this branch, so I proposed no precision changes, just the conv guard.

For the open issues from the #230 discussion I left notes in #243: component-seam parity against the diffusers reference as a precision-robust quality metric (happy to port that harness from this PR as a follow-up), plus quant sensitivity data (depth decoder is the fragile one, embeddings tolerate Q8, and slicing the LM head to the 16385 sampleable rows saves ~1.5 GB for free).

Feel free to close this PR in favor of the preview branch, or cherry-pick anything useful from it; the converted packages from this PR live at https://huggingface.co/joemattie/MiniMax-Music3-GGUF if any variants help.

[JoeMattie]

Closing in favor of the official port on preview/minimax-music-3 plus #243. The branch and the HF package stay up for cherry-picking.

[0xShug0]

@JoeMattie Thanks! I added your repo to the HF README. I will include your repo as a download source in the spec once we finalize it!

[mirek190]

0xShug0

@JoeMattie Thank you for the PR! We quietly made the model public this morning but haven’t announced it yet. https://github.com/0xShug0/audio.cpp/tree/preview/minimax-music-3 and GGUFs are at https://huggingface.co/audio-cpp/MiniMax-Music3-GGUF.

Would you like to give it a try? I think the best way is to apply your improvements or pieces we’re missing to preview/minimax-music-3 and hopefully we can address the issues mentioned here #230 (comment).

I really appreciate you taking the time to work on Music 3. And definitely don’t let this discourage you from contributing more models!

Why is there is no Q8 versions?
https://huggingface.co/audio-cpp/MiniMax-Music3-GGUF/tree/main

[0xShug0]

@mirek190 I don’t see a clear win for q8_0 over q4_0 in terms of speed or VRAM usage. The q4_k component GGUFs in the repo are just sync leakage. The final repo may just keep q4_0, or whichever variant ends up winning.

[mirek190]

@mirek190 I don’t see a clear win for q8_0 over q4_0 in terms of speed or VRAM usage. The q4_k component GGUFs in the repo are just sync leakage. The final repo may just keep q4_0, or whichever variant ends up winning.

Q4 has a low precision comparing to Q8. ( quality should be better )
Cards 24GB + should handle it

So we could allow to use Q4 , Q8 and bf16 versions

[0xShug0]

@mirek190 I don’t see a clear win for q8_0 over q4_0 in terms of speed or VRAM usage. The q4_k component GGUFs in the repo are just sync leakage. The final repo may just keep q4_0, or whichever variant ends up winning.

Q4 has a low precision comparing to Q8. ( quality should be better ) Cards 24GB + should handle it

So we could allow to use Q4 , Q8 and bf16 versions

okay sounds good. I will add them later.