Skip to content

fix(export): keep multiplier depthwise convs FP32 so the MobileCLIP image encoder builds as FP16 - #244

Merged
davidamacey merged 1 commit into
mainfrom
fix/224-mobileclip-fp16
Oct 11, 2026
Merged

davidamacey merged 1 commit into
mainfrom
fix/224-mobileclip-fp16

Conversation

@davidamacey

Copy link
Copy Markdown
Owner

Closes #224

Cause

The failing layers are depthwise convs with a channel multiplier: the three stages.N/downsample/proj/proj.0/reparam_conv convs (7x7, stride 2, 2 outputs per group) and final_conv/reparam_conv (3x3, 640 groups, 1280 outputs). TensorRT 11.1 (strongly typed) has no FP16 kernel for a grouped conv with one input channel per group and more outputs than groups. Its own warning says so: "No low-precision conv kernel available for this strongly-typed Conv ... Consider keeping Conv ... in FP32". The weights are already reparameterised (single conv per layer), so the exporter is not at fault. Blocking only the three downsample convs moves the failure to final_conv; blocking all four builds.

Fix

bake_fp16_onnx passes the multiplier depthwise convs (found from initializer weight shapes) as node_block_list, so they stay FP32 and the other 185 convs are FP16. The existing reconcile pass inserts the casts. Offline test added; it failed before the change (dw_w came out FP16) and passes after.

Evidence (GPU 1 only, RTX 3080 Ti, zero foreign utilisation, one-shot containers, nothing left running)

  • Engine built by export_mobileclip_image_encoder.py itself (Triton-image TensorRT 11.1, FP16, no fallback): 71.5 MB, 72 s. trtexec FP16: 75.1 MB, 65 s. FP32: 146.6 MB, 62 s.
  • Parity vs FP32 on 200 pinned COCO images (hash-verified): cosine min 0.99984, mean 0.99998; top-1 neighbour agreement 99.5 percent, top-5 overlap 99.3 percent.
  • Engine-only trtexec throughput, FP32 vs FP16: batch 8 627 vs 966 images/s (12.7 vs 8.3 ms), batch 32 771 vs 1604 images/s (41.5 vs 19.9 ms). perf_analyzer through Triton was not run.
  • Docs: PERFORMANCE.md section, CHANGELOG, optimization plan checkbox.

Not changed

model_setup.sh records a precision only for pe_image_encoder; MobileCLIP has no ledger entry, so there is nothing that could report fp16 wrongly. The exporter's printed FP32 fallback stays for any future build failure.

pre-commit run --from-ref origin/main --to-ref HEAD exit 0. The bake tests need onnxconverter-common, which the repo venv lacks (they skip there, as before); I ran them with it installed to a scratch directory (PYTHONPATH): 18 passed.

…mage encoder builds as FP16

TensorRT 11.1 has no FP16 kernel for a grouped conv with one input channel
per group and more outputs than groups, so the build stopped at the
reparam_conv downsample layers and the exporter fell back to FP32. The FP16
bake now leaves those convs in FP32.
@davidamacey
davidamacey merged commit 55eb73f into main Oct 11, 2026
5 checks passed
@davidamacey
davidamacey deleted the fix/224-mobileclip-fp16 branch October 11, 2026 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MobileCLIP image encoder FP16 build fails on TensorRT 11.1 (reparam conv), exporter ships FP32

1 participant