convert-hf-to-gguf-bitnet.py --outtype i2_s silently writes F16 (not I2_S) for LlamaForCausalLM BitNet checkpoints (Falcon3 / Falcon-E 1.58bit)

[claudioaldrighettiatroos]

Summary

utils/convert-hf-to-gguf-bitnet.py accepts --outtype i2_s for every architecture, but the I2_S packing code path exists only in BitnetModel (architecture BitNetForCausalLM). For the LlamaForCausalLM BitNet checkpoints listed as supported in the README (tiiuae/Falcon3-*-1.58bit, tiiuae/Falcon-E-*), LlamaModel.write_tensors unpacks the offline-quantized ternary weights and stores them as F16, without any warning. The result is a valid but full-size GGUF (14.9 GB for Falcon3-7B instead of ~2.7 GB) with no I2_S tensors, while the metadata still claims an I2_S file type.

Since llama-quantize in this tree has no I2_S ftype either (#619), there is currently no way to obtain an I2_S GGUF for the Falcon 1.58-bit models from their HF checkpoints.

Environment

• microsoft/BitNet at 0b341e5 (current main), submodule 3rdparty/llama.cpp at 390c3077
• macOS 26.5.2, Apple M2 Pro, Python 3.14.7, gguf installed from 3rdparty/llama.cpp/gguf-py (as done by setup_env.py)

Steps to reproduce

hf download tiiuae/Falcon3-7B-Instruct-1.58bit --local-dir models/Falcon3-7B-Instruct-1.58bit
python utils/convert-hf-to-gguf-bitnet.py models/Falcon3-7B-Instruct-1.58bit --outtype i2_s

Converter log (every linear weight):

INFO:hf-to-gguf:blk.0.ffn_down.weight,       torch.uint8 --> F16, shape = {23040, 3072}
INFO:hf-to-gguf:blk.0.attn_q.weight,         torch.uint8 --> F16, shape = {3072, 3072}
...
INFO:gguf.gguf_writer:models/Falcon3-7B-Instruct-1.58bit/ggml-model-i2_s.gguf: n_tensors = 255, total_size = 14.9G

Resulting file (read with gguf.GGUFReader): tensor types {F16: 198, F32: 57}, zero I2_S tensors, general.file_type = 40. The same converter run on microsoft/bitnet-b1.58-2B-4T-bf16 correctly produces 210 I2_S tensors, so the difference is the architecture, not the input format (the Falcon checkpoint is offline-quantized: uint8 packed weights + weight_scale tensors, 196 of them, which LlamaModel does unpack correctly).

Root cause

• LlamaModel.write_tensors (utils/convert-hf-to-gguf-bitnet.py, from line 776) handles the offline-quantized weights (unpack at ~line 800, scale_map), but its quantization dispatch only has TL1 and TL2 branches (lines ~869-878) followed by else: # default to float16 for quantized tensors (~line 880).
• The I2_S branch (quantize_to_i2_s(data, override_scale=...)) exists only in BitnetModel.write_tensors (line 1164).
• ftype_map / --outtype (lines 1216, 1237) accept i2_s regardless of the model class, so the request is silently downgraded to F16.

Suggested fix

Port the I2_S branch from BitnetModel.write_tensors to LlamaModel.write_tensors (the ternary values and scale_map are already available there, so quantize_to_i2_s(data, override_scale=scale) is a small change), or make the converter fail loudly when --outtype i2_s is requested for a class that cannot produce it. Related: #619 (conversion flow / missing I2_S in llama-quantize), #550 (Falcon3 TL2 support in setup_env.py), #616 (LlamaModel dequantization), #620 (general.file_type value).