Repository navigation
fix(export): keep multiplier depthwise convs FP32 so the MobileCLIP image encoder builds as FP16 - #244
Merged
Conversation
…mage encoder builds as FP16 TensorRT 11.1 has no FP16 kernel for a grouped conv with one input channel per group and more outputs than groups, so the build stopped at the reparam_conv downsample layers and the exporter fell back to FP32. The FP16 bake now leaves those convs in FP32.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #224
Cause
The failing layers are depthwise convs with a channel multiplier: the three
stages.N/downsample/proj/proj.0/reparam_convconvs (7x7, stride 2, 2 outputs per group) andfinal_conv/reparam_conv(3x3, 640 groups, 1280 outputs). TensorRT 11.1 (strongly typed) has no FP16 kernel for a grouped conv with one input channel per group and more outputs than groups. Its own warning says so: "No low-precision conv kernel available for this strongly-typed Conv ... Consider keeping Conv ... in FP32". The weights are already reparameterised (single conv per layer), so the exporter is not at fault. Blocking only the three downsample convs moves the failure tofinal_conv; blocking all four builds.Fix
bake_fp16_onnxpasses the multiplier depthwise convs (found from initializer weight shapes) asnode_block_list, so they stay FP32 and the other 185 convs are FP16. The existing reconcile pass inserts the casts. Offline test added; it failed before the change (dw_wcame out FP16) and passes after.Evidence (GPU 1 only, RTX 3080 Ti, zero foreign utilisation, one-shot containers, nothing left running)
export_mobileclip_image_encoder.pyitself (Triton-image TensorRT 11.1, FP16, no fallback): 71.5 MB, 72 s.trtexecFP16: 75.1 MB, 65 s. FP32: 146.6 MB, 62 s.trtexecthroughput, FP32 vs FP16: batch 8 627 vs 966 images/s (12.7 vs 8.3 ms), batch 32 771 vs 1604 images/s (41.5 vs 19.9 ms).perf_analyzerthrough Triton was not run.Not changed
model_setup.shrecords a precision only forpe_image_encoder; MobileCLIP has no ledger entry, so there is nothing that could reportfp16wrongly. The exporter's printed FP32 fallback stays for any future build failure.pre-commit run --from-ref origin/main --to-ref HEADexit 0. The bake tests needonnxconverter-common, which the repo venv lacks (they skip there, as before); I ran them with it installed to a scratch directory (PYTHONPATH): 18 passed.