From 342c5564a91bfcd7a0192cd8ec59067a5ae59d5a Mon Sep 17 00:00:00 2001 From: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com> Date: Sat, 5 Sep 2026 02:49:08 -0700 Subject: [PATCH] Fix wrong general.file_type value for I2_S GGUF conversion output Motivation: utils/convert-hf-to-gguf-bitnet.py and utils/convert-bitnet-embedding-to-gguf.py both wrote general.file_type = 40 for I2_S-quantized output. Per llama.cpp's llama_ftype enum, 40 is LLAMA_FTYPE_MOSTLY_Q1_0 and 41 is LLAMA_FTYPE_MOSTLY_I2_S, so tools that read this metadata field (e.g. llama-cli) report the quant type as Q1_0 instead of I2_S. This does not affect the tensor data itself or the model's numerical behavior -- only the general.file_type metadata field and anything that displays/labels the quant type from it. Approach: Change the hardcoded literal from 40 to 41 in both converters, and drop a stale comment ("matches official model") on the now-corrected line that implied 40 was intentional. Two sibling converters (utils/convert.py, utils/convert-ms-to-gguf-bitnet.py) write general.file_type for their own "i2" outtype from a different, pre-existing GGMLFileType.MostlyI2 enum (value 2, not 40/41). That is a separate, out-of-scope potential mislabeling in a different code path and is not touched here. Validation: The 3rdparty/llama.cpp git submodule could not be checked out in this environment (git submodule update --init failed with "No space left on device"), and the torch/gguf/sentencepiece Python packages this repo's converters import are not installed here either, so the full HF-to-GGUF pipeline could not be run end to end. To still validate by execution rather than inspection alone: - For convert-hf-to-gguf-bitnet.py: stubbed sys.modules for torch, sentencepiece, safetensors, and gguf (the gguf stub's MODEL_ARCH / GGMLQuantizationType return the attribute name for any access, and its GGUFWriter just records add_file_type's argument), loaded the real, unmodified utils/convert-hf-to-gguf-bitnet.py via importlib.util.spec_from_file_location, built a BitnetModel via object.__new__ with ftype set to the I2_S sentinel, and called the real Model.set_gguf_parameters() method on it. Before the fix this recorded file_type 40; after changing 40 to 41 in the source, rerunning the same harness recorded file_type 41. - For convert-bitnet-embedding-to-gguf.py: read the exact ftype-selection if/elif/else block out of the file at runtime and exec'd that extracted source text (unmodified from disk) with args.outtype = "i2_s"; the resulting ftype was 40 before the fix and 41 after. - python3 -m py_compile on both changed files passes. - Verified independently via the GitHub API that this repo's pinned llama.cpp submodule commit (390c307752ab78fd8189f359d6954c9ba1be74af) defines LLAMA_FTYPE_MOSTLY_Q1_0 = 40 and LLAMA_FTYPE_MOSTLY_I2_S = 41 in include/llama.h, confirming 41 is the correct value. Not run: the actual HF checkpoint -> GGUF conversion end to end, and llama-quantize/llama-cli against a real model (blocked by the submodule checkout failure above). Report: https://github.com/microsoft/BitNet/issues/619 Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com> Assisted-by: claude-sonnet-5 (via Claude Code) --- utils/convert-bitnet-embedding-to-gguf.py | 2 +- utils/convert-hf-to-gguf-bitnet.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/utils/convert-bitnet-embedding-to-gguf.py b/utils/convert-bitnet-embedding-to-gguf.py index 3a9e48924..fe479d25f 100644 --- a/utils/convert-bitnet-embedding-to-gguf.py +++ b/utils/convert-bitnet-embedding-to-gguf.py @@ -535,7 +535,7 @@ def main(): elif args.outtype == "f16": ftype = 1 # GGML F16 else: # i2_s - ftype = 40 # LLAMA_FTYPE_MOSTLY_I2_S + ftype = 41 # LLAMA_FTYPE_MOSTLY_I2_S logger.info(f"Converting {dir_model.name} (arch={arch}) to GGUF ({args.outtype})") diff --git a/utils/convert-hf-to-gguf-bitnet.py b/utils/convert-hf-to-gguf-bitnet.py index b11e831b9..3d3e795a9 100644 --- a/utils/convert-hf-to-gguf-bitnet.py +++ b/utils/convert-hf-to-gguf-bitnet.py @@ -156,7 +156,7 @@ def set_gguf_parameters(self): # Map ggml tensor type to llama ftype for general.file_type metadata ftype_val = self.ftype if self.ftype == gguf.GGMLQuantizationType.I2_S: - ftype_val = 40 # LLAMA_FTYPE_MOSTLY_I2_S (matches official model) + ftype_val = 41 # LLAMA_FTYPE_MOSTLY_I2_S self.gguf_writer.add_file_type(ftype_val) logger.info(f"gguf: file type = {ftype_val}")