fp8-quantization

Here are 7 public repositories matching this topic...

MicroPerceptron / virtio-accel

Transport-neutral, vendor-agnostic virtual accelerator devices. First class cross-platform NPU backend support for Apple, Intel, AMD, Qualcomm devices, as well as Vulkan for cross-platform GPU support.

accelerator vulkan virtualization hardware-acceleration mesa virtio coreml coreml-framework heterogeneous-computing openvino npu inference-acceleration neural-engine xdna qualcomm-hexagon fp8-quantization

Updated Oct 23, 2026
Rust

Crynge / SpecInferKit

Star

Production-grade speculative decoding toolkit for LLMs — multi-algorithm, FP8 quantization, online training, distributed serving, and comprehensive evaluation suite.

python nlp benchmarking machine-learning deep-learning eagle pytorch medusa gpu-acceleration distributed-training model-acceleration inference-optimization ai-inference llm-serving llm-inference speculative-decoding fp8-quantization draft-models

Updated Jul 1, 2026
Python

fattorib / diffusion-profiled

Star

Companion code for https://benfattori.com/profiled-ddpm/

pytorch triton cifar10 diffusion-models ddpm fp8-quantization

Updated Oct 25, 2026
Python

jojo535 / llm-inference-rtx5070ti

Star

LLM Inference Optimization for RTX 5070 Ti (Blackwell) - Benchmarks, quantization, serving configs for vLLM/SGLang/TensorRT-LLM/llama.cpp

docker benchmarking cuda nvidia-gpu awq llama-cpp vllm gptq llm-inference gguf tensorrt-llm sglang blackwell-architecture fp8-quantization rtx-5070-ti

Updated Aug 26, 2026
Python

Keerthanareddy17 / foveated-vlm-search

Star

Developed a foveated VLM search system using Qwen3-VL-2B, LoRA, vLLM, and FP8 quantization. Benchmarked 4 search strategies on 500 COCO queries; fine-tuning achieved 85.6% accuracy with 1.1% false positives. Reduced vision tokens 17.2x and latency 45% on 1920×1440 images using adaptive thumbnail-and-crop search.

computer-vision lora fine-tuning coco-dataset vllm qwen3-vl fp8-quantization foveated-vision