Code for https://arxiv.org/pdf/2608.16157
-
Updated
Sep 7, 2026 - Python
#
gpu-offloading
Here are 3 public repositories matching this topic...
Train and serve MoE models that do not fit in VRAM: fused 4-bit experts, QLoRA, CPU/NVMe offload, and fast inference on consumer NVIDIA GPUs.
cuda transformers pytorch triton moe lora quantization nvme fine-tuning mixture-of-experts int4 fp8 llm-training llm-inference qlora bitsandbytes nf4 consumer-gpu mxfp4 gpu-offloading
-
Updated
Sep 9, 2026 - Python
Triton kernels for 4-bit MoE inference: grouped NF4/MXFP4 GEMM, INT4 GEMV, FP8 paged attention, and CPU/NVMe expert streaming.
cuda pytorch triton moe quantization nvme nvidia-gpu model-serving gpu-kernels mixture-of-experts int4 fp8 llm-inference bitsandbytes paged-attention nf4 mxfp4 grouped-gemm w4a16 gpu-offloading
-
Updated
Sep 9, 2026 - Python
Add this topic to your repo
To associate your repository with the gpu-offloading topic, visit your repo's landing page and select "manage topics."