voxtral.cpp

voxtral.cpp is a C++17 implementation of Mistral AI's Voxtral speech recognition models. It can transcribe WAV files, expose batch and realtime network APIs, or be embedded through a stable C API.

Official prebuilt binaries are not currently published. Build the project from source for your system and hardware. Model weights are also separate downloads.

Features

• Offline file transcription with the voxtral command.
• Realtime and offline Voxtral GGUF model support.
• CPU inference and a Vulkan backend.
• HTTP batch transcription and WebSocket realtime transcription.
• A versioned C API with polling, bounded backpressure, cancellation, reset, and metrics.
• A local browser playground for microphone and WAV input.
• A GGUF conversion script and a Voxtral-aware quantizer.

Requirements

The core build needs:

• CMake 3.16 or newer for CPU builds.
• A C++17 compiler.
• Git and a build tool such as Ninja.
• Network access during the default CMake configure step, because CMake fetches the repository's pinned GGML revision.

A Vulkan build also needs CMake 3.19 or newer, Vulkan development files, a runtime Vulkan loader and driver, and glslc at build time. The server additionally needs Boost 1.75 or newer and the Boost.JSON library.

The commands below use Ubuntu 24.04 package names. Package names differ on other distributions. See the build guide for Linux and NixOS instructions.

Quick start

This Vulkan example installs the Ubuntu 24.04 dependencies, clones the repository, builds the command-line tools, downloads a converted realtime GGUF, and transcribes a 16 kHz WAV file:

sudo apt update && sudo apt install build-essential cmake ninja-build git curl libvulkan-dev glslc

git clone https://github.com/MrShitFox/voxtral.cpp.git && cd voxtral.cpp

cmake -S . -B build-vulkan -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DGGML_VULKAN=ON -DVOXTRAL_BUILD_SERVER=OFF -DBUILD_TESTING=OFF && cmake --build build-vulkan --parallel

./tools/download_model.sh Q4_K_M --model realtime

./build-vulkan/voxtral --model models/voxtral/Q4_K_M.gguf --audio /path/to/audio.wav --gpu vulkan

For a CPU-only build in the same checkout, run:

cmake -S . -B build-cpu -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DGGML_VULKAN=OFF -DVOXTRAL_BUILD_SERVER=OFF -DBUILD_TESTING=OFF && cmake --build build-cpu --parallel

./build-cpu/voxtral --model models/voxtral/Q4_K_M.gguf --audio /path/to/audio.wav --gpu none

Build

The normal executable build links the project libraries statically into the tools. It is not a completely static Linux binary: system runtime libraries remain dynamic.

CPU:

cmake -S . -B build-cpu -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DGGML_VULKAN=OFF -DVOXTRAL_BUILD_SERVER=OFF -DBUILD_TESTING=OFF && cmake --build build-cpu --parallel

Vulkan:

cmake -S . -B build-vulkan -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DGGML_VULKAN=ON -DVOXTRAL_BUILD_SERVER=OFF -DBUILD_TESTING=OFF && cmake --build build-vulkan --parallel

The Vulkan build is compiled for the local system. At runtime it uses the installed Vulkan loader and GPU driver. Build on the machine where the binaries will be used.

For server, shared-library, install, NixOS, and troubleshooting instructions, read Building voxtral.cpp.

Download a model

The download script retrieves converted GGUF files; it does not download the original safetensors checkpoint:

./tools/download_model.sh Q4_K_M --model realtime

The default output is models/voxtral/Q4_K_M.gguf. Use --model mini for the offline Voxtral Mini 3B family.

No model weights are included in project binaries or source archives. See Models for repositories, variants, conversion, quantization, storage, and model-license information.

Transcribe a file

Use a 16 kHz WAV input. Select --gpu vulkan for a Vulkan build or --gpu none for CPU:

./build-vulkan/voxtral --model models/voxtral/Q4_K_M.gguf --audio /path/to/audio.wav --gpu vulkan

The transcript and token IDs are written to standard output; logs and a runtime backend summary are written to standard error. The complete argument and exit reference is in Command-line reference.

Start the server

The server is a Vulkan application. It requires a shared libvoxtral, Boost headers, and Boost.JSON:

cmake -S . -B build-server -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON -DGGML_VULKAN=ON -DVOXTRAL_BUILD_SERVER=ON -DBUILD_TESTING=OFF && cmake --build build-server --parallel

For local development without authentication:

./build-server/voxtral-server --model models/voxtral/Q4_K_M.gguf --listen 127.0.0.1 --port 8080 --no-auth

For an authenticated local server:

VOXTRAL_SERVER_API_KEY_FILE="$HOME/.config/voxtral/api-key" ./build-server/voxtral-server --model models/voxtral/Q4_K_M.gguf --listen 127.0.0.1 --port 8080

Read Voxtral server before binding to a non-loopback address. It documents authentication, HTTP requests, WebSocket messages, concurrency, backpressure, errors, and shutdown.

Playground

The playground is a local Node.js gateway and dependency-free browser client. It keeps the server API key out of browser code and supports microphone and WAV input. Node.js 20.11 or newer is required.

cd examples/playground && npm ci

cd examples/playground && npm start

Open http://127.0.0.1:3000. If the upstream server uses authentication, set VOXTRAL_API_KEY in the playground process. See Voxtral Playground.

C API

The public headers are include/voxtral.h and include/voxtral-stream.h. Build the shared library with:

cmake -S . -B build-shared -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON -DGGML_VULKAN=OFF -DVOXTRAL_BUILD_SERVER=OFF -DBUILD_TESTING=OFF && cmake --build build-shared --parallel

The build produces libvoxtral.so.1.1.0 with SONAME libvoxtral.so.1 on Linux, plus the voxtral-streaming-c example. Run the example with a realtime model and a mono PCM16 16 kHz WAV:

./build-shared/voxtral-streaming-c models/voxtral/Q4_K_M.gguf /path/to/audio.wav 480

See C API for lifecycle, versioning, backpressure, events, metrics, struct layouts, and all exported functions.

Documentation

• Build guide
• Models, conversion, and quantization
• Command-line reference
• HTTP and WebSocket server
• Public C API
• Playground
• Node.js developer tests
• Realtime diagnostics

Supported models

• Voxtral Mini 4B Realtime 2602 GGUF, with GGUF architecture voxtral_realtime, for incremental transcription.
• Voxtral Mini 3B 2507 GGUF, with GGUF architecture voxtral, for offline transcription.

Use a realtime model for the WebSocket server, playground realtime mode, or the streaming C API. The file CLI and batch endpoint also accept the offline family.

Current platform status

Linux CPU and Linux Vulkan builds are maintained. The tested Vulkan configuration is an AMD Radeon RX 6600 using Vulkan/RADV. Other Vulkan devices may work with an installed vendor driver, but are not claimed as tested here.

The documented build targets are CPU and Vulkan. CUDA is not a validated or supported documentation target in this project.

Build locally for the intended operating system and hardware. In particular, NixOS-built binaries commonly reference /nix/store and should not be copied to arbitrary non-NixOS Linux systems.

License

The source code is licensed under the MIT License. Model weights have their own licenses and terms; review the model repository before downloading or redistributing a model.