quantiz

Here are 2 public repositories matching this topic...

2bit quants, kernels and runtime for consumer devices.

From-scratch C++ LLM inference engine — runs SmolLM2 (135M–1.7B) on CPU with hand-written NEON SIMD, INT8 quantization, and a live web UI.

cpp neon inference simd transformer quantization infere quantiz apple-silicon llm

To associate your repository with the quantiz topic, visit your repo's landing page and select "manage topics."