Gemma4 ov int4 and stateful fixes by cavusmustafa · Pull Request #260 · ravi9/llama.cpp

cavusmustafa · 2026-07-18T00:22:35Z

enables gemma4 dense stateful support (limited support, up to 1k context)
optional int4 downscale for requant: sacrifice accuracy for performance (env to enable - GGML_OPENVINO_INT4_REQUANT)
optional disk spill when weight loading to reduce initial RSS peak for low memory systems (env to enable: GGML_OPENVINO_SPILL_DIR)

…aved-SWA models

…uant target

… disk

cavusmustafa force-pushed the gemma4_ov_int4_and_stateful_fixes branch from 530aa62 to cbb8242 Compare July 21, 2026 19:26

wine99 force-pushed the dev_backend_openvino branch from 4250de7 to 175f422 Compare July 27, 2026 04:33

cavusmustafa force-pushed the gemma4_ov_int4_and_stateful_fixes branch from 53d099b to 61c08f4 Compare August 11, 2026 21:16

wine99 force-pushed the dev_backend_openvino branch 2 times, most recently from 37b164f to 2bacf9e Compare August 14, 2026 05:24

cavusmustafa added 4 commits August 14, 2026 19:40

ggml-openvino: fix stateful decode for Gemma-4 per-layer-type head sizes

940a989

ggml-openvino: fix MSVC narrowing error in permute

c50dfcc

ggml-openvino: classify sliding-window layers structurally on interle…

9d08d82

…aved-SWA models

ggml-openvino: add GGML_OPENVINO_REQUANT_KQUANT to select a 4-bit req…

4acdec3

…uant target

cavusmustafa force-pushed the gemma4_ov_int4_and_stateful_fixes branch from 61c08f4 to 4acdec3 Compare August 14, 2026 18:07

ggml-openvino: add GGML_OPENVINO_SPILL_DIR to spill weight buffers to…

cfcaa44

… disk

wine99 force-pushed the dev_backend_openvino branch from 678ba21 to 16f69ac Compare August 18, 2026 01:01

ravi9 force-pushed the dev_backend_openvino branch from eb54a50 to 04b5691 Compare August 18, 2026 14:43

Navigation Menu