Port model-ops postprocess fixes (0.0.6) and support YOLOX / DAMO-YOLO by parkjinman98 · Pull Request #22 · mobilint/mblt-vision-python

parkjinman98 · 2026-09-28T07:01:05Z

Three commits:

4f11b0e fix(post): port mblt-model-ops postprocess fixes (0.0.6). I compared each task's decode path against mblt-model-ops origin/master since the 0.0.4 sync and ported the fixes this repo lacked.
57175fe feat: support the YOLOX and DAMO-YOLO detection families. 13 models, configured from mblt-model-ops' pipeline.yaml on branch jm/temp at 0368367e8.
3d93f7f chore: link agent guides and skills to one copy each. This follows mblt-model-ops' layout: .agents/skills/<name> are now symlinks to ../../.claude/skills/<name>, and CLAUDE.md is a symlink to AGENTS.md. The skill copies were identical, and every fact in CLAUDE.md's summary was already in AGENTS.md, so no content is lost.

1. Postprocess fixes (0.0.6)

non_max_suppression: zero-area boxes. The function now suppresses a box only when its IoU is above the threshold, as torchvision.ops.nms does. Two zero-area boxes give 0/0 = NaN, and the old <= test read that NaN as an overlap and dropped the box. model-ops keeps these boxes too.
ImageNet top-1/top-5 ranking. Both now come from one stable ranking, which matches model-ops' evaluator. On ties the higher class index wins, and top-1 is always the head of top-5. Before, argmax and topk could disagree on tied quantized scores.
nmsout2eval_pose copies before rescaling. It used to rewrite the caller's NMS keypoints in place; model-ops already copies.
Two stale test_wrapper pose tests fixed. They expected raw decode-true visibility, but the code applies a sigmoid, as the postprocess-hierarchy tests pin.

Not ported, deliberately:

model-ops' stable NMS candidate sorts. I kept Ultralytics' unstable argsort(descending=True), which reproduces Ultralytics' tie order. On 500 COCO val images the stable sort moved mAP50-95 by −0.00047 (YOLOv8m), −0.0038 (YOLOv8m-pose) and +0.00029 (YOLOv8m-seg), entirely through tied scores.
Places where model-ops is the worse side:
- IoU NMS on YOLO26 heads.
- A second NMS on face output that is already NMS-free.
- Integer-truncated mask crops.

Measured on aries-rb MXQ:

model	set	before	after
YOLOv8m, YOLOv8m-pose, YOLOv8m-seg	COCO val, 500 images	—	predictions byte-identical to `main`
ResNet50	ImageNet, all 50,000 images	top-1 0.80474, top-5 0.95280	top-1 0.80524, top-5 0.95260

The ImageNet change comes from the tie ranking alone. mblt_vision.__version__ is bumped to 0.0.6 so model-ops' parity pin can move to it.

2. YOLOX and DAMO-YOLO

Models:

YOLOX Nano, Tiny, s, m, l, x and Darknet53.
DAMO-YOLO T, S and M, each with its distilled variant.

DAMO-YOLO L, L-distill and the Nano models are left out because they have no public checkpoint, as in model-ops. No Hub repository exists yet, so every YAML sets file_cfg.local_artifact_only: true and needs a local model_path.

Postprocessing. post_cfg.head (yolox or damoyolo) selects a decoder that only turns the raw head into candidates. The anchorless filter, NMS and inverse letterbox are reused as they are. An unknown head, or a head on another task, raises an error.

YOLOX decodes one (batch, anchors, 5 + nc) tensor: xy = (raw + grid) * stride, wh = exp(raw) * stride, score = objectness × class.
DAMO-YOLO decodes six per-level maps. reg_max: 16 means 17 bins.
Neither adds the half-cell offset that Ultralytics uses.
DAMO-YOLO layout. At a 640 input the stride-8 class map is 80×80×80, so the tensor alone can't say which axis holds the classes. The NCHW/NHWC layout is therefore decided once for the whole set of heads, and a mixed set raises an error.

Preprocessing:

LetterBox.center and padding_value. YOLOX anchors the image top-left with 114; DAMO-YOLO anchors it top-left with zeros.
Reader.color_mode. YOLOX expects BGR, which I checked against upstream: cv2.imread with no conversion, and ValTransform(legacy=False). Feeding RGB costs 4.9 mAP50-95 on 500 images.
No Normalize step. Both models take the unscaled image, so their pipelines declare no Normalize, and the ONNX path casts the byte tensor to the float type the graph declares.
Top-left geometry. Wherever geometry is derived from image shapes alone (PostBase.ratio_pads_for and Results plotting), it now follows the model's letterbox_center(pre_cfg). For centered models nothing changes.

Verified end to end on the full COCO val2017 split, using ONNX exports of the upstream checkpoints made with model-ops' exporters (CUDA execution provider):

model	mAP50-95 (this package)	upstream
YOLOX-s	40.679	40.5
DAMO-YOLO-T	41.789	41.8

Validation

pytest tests -m "not requires_network and not requires_npu": 812 passed, 0 failed.
- This includes 4 new postprocess-fix regressions. Each one fails against the old code.
- It also includes 23 YOLOX/DAMO-YOLO tests. They check both decodes against upstream's formulas in every accepted layout, push a planted box through NMS, and cover anchoring, BGR input, the ONNX cast, dispatch errors and the top-left inverse geometry.
ruff format and ruff check at 120 columns are clean on the new files, and git diff --check is clean.
AGENTS.md, CLAUDE.md, both mblt-vision skill copies (diffed identical) and mblt_vision/README.md are updated.

Not verified:

No YOLOX or DAMO-YOLO MXQ artifact exists yet, so there are no NPU scores. NHWC head handling is covered by unit tests only.
Only YOLOX-s and DAMO-YOLO-T were scored end to end.

🤖 Generated with Claude Code

Compared every task's decode path against mblt-model-ops origin/master since the 0.0.4 sync and ported what mblt-vision lacked: - non_max_suppression suppresses only IoU above the threshold, as torchvision.ops.nms does. Two zero-area boxes gave 0/0 = NaN, which the old `<=` test read as an overlap and dropped (model-ops keeps them). - ImageNet top-1/top-5 come from one stable ranking, matching model-ops' evaluator: ties favour the higher class index and top-1 is always the head of top-5. argmax and topk could disagree on tied quantized scores. - nmsout2eval_pose copies before scale_coords rescales in place; it was rewriting the caller's NMS keypoints (model-ops copies). - The two stale test_wrapper pose tests expected raw decode-true visibility; the code sigmoids it, as the postprocess-hierarchy tests pin. Not ported: model-ops' stable NMS candidate sorts. Kept Ultralytics' unstable argsort, which reproduces its tie order; the stable sort moved YOLOv8m -0.00047, YOLOv8m-pose -0.0038, YOLOv8m-seg +0.00029 mAP50-95 on 500 COCO val images, entirely through ties. Also not ported where model-ops is the worse side (IoU NMS on YOLO26 heads, second NMS on NMS-free face output, integer-truncated mask crops). Measured on aries-rb MXQ: - COCO 500 images: YOLOv8m, YOLOv8m-pose, YOLOv8m-seg predictions are byte-identical to main (the NMS fix and pose copy change nothing there). - ImageNet ResNet50, 50000 images: top-1 0.80474 -> 0.80524, top-5 0.95280 -> 0.95260, from the tie ranking alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Adds 13 object-detection models -- YOLOX Nano/Tiny/s/m/l/x/Darknet53 and DAMO-YOLO T/S/M with their distilled variants -- configured from mblt-model-ops' pipeline.yaml (branch jm/temp, 0368367e8). DAMO-YOLO L, L-distill and the Nano models have no public checkpoint and are left out, as they are there. No Hub repository exists yet, so every YAML keeps file_cfg.local_artifact_only: true and takes a local model_path. Postprocessing: post_cfg.head (yolox | damoyolo) selects a decoder that only turns the raw head into candidates; the anchorless filter, NMS and inverse letterbox are reused. YOLOX decodes one (batch, anchors, 5 + nc) tensor, (raw + grid) * stride and exp(raw) * stride, objectness x class. DAMO-YOLO decodes six per-level maps; reg_max 16 means 17 bins. Neither adds the Ultralytics half-cell offset. The 640-input stride-8 DAMO class map is 80x80x80 and cannot name its own channel axis, so the head set's NCHW/NHWC layout is resolved jointly and a mixed set fails loudly. Any other head value, or a head on another task, is rejected. Preprocessing: LetterBox gains center and padding_value (YOLOX anchors top-left with 114, DAMO-YOLO with zeros), and Reader gains color_mode (YOLOX reads BGR; arrays are flipped into a copy). Both take the unscaled image, so their pipelines declare no Normalize and the ONNX path casts the byte tensor to the graph's float dtype. Geometry derived from shapes alone -- PostBase.ratio_pads_for and Results plotting -- now follows the model's letterbox_center(pre_cfg); centered models pass through unchanged. Verified end to end on the full COCO val2017 split through ONNX exports of the upstream checkpoints (model-ops' exporters, CUDA EP): YOLOX-s 40.679 mAP50-95 against upstream's 40.5, DAMO-YOLO-T 41.789 against 41.8. Unit tests pin both decodes to upstream's formulas in every accepted layout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

chatgpt-codex-connector · 2026-09-28T07:01:10Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

parkjinman98 · 2026-09-28T07:02:08Z

@mobilint-review Could you review this PR?

github-actions

Codex review

Requested by @parkjinman98.

Verdict

I found one correctness issue in the DAMO-YOLO decoder: valid float16 outputs can fail during distribution decoding before NMS.

Suggested next steps

Make the distribution projection dtype-compatible with backend outputs.
Add a DAMO-YOLO float16 decode test covering the full postprocessing path.

Trigger: issue_comment
Source: #22 (comment)

Follow mblt-model-ops' layout so no guide or skill exists twice: - .agents/skills/<name> are symlinks to ../../.claude/skills/<name>. The two copies of mblt-vision and mblt-vision-readme were identical, so no content moves. - CLAUDE.md is a symlink to AGENTS.md, the canonical guide. Its wrapper summary restated facts AGENTS.md already holds; the one pointer it alone carried (read the mblt-vision skill) now opens AGENTS.md. AGENTS.md's documentation rule now describes the links instead of asking for every skill edit to be written to both copies and diffed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Review of #22 found that a float16 DAMO-YOLO head fails the bin projection: `probabilities @ bin_values` multiplies the backend dtype by float32 bin values, and matmul does not promote, so it raises "expected scalar type Half but found Float". YOLOX had the matching silent case: `exp` of a float16 size regression overflows to inf above ~11 before the float32 grid can promote it, so an extreme box became non-finite. Both decoders now upcast their raw heads to float32 on entry. Casting the bin vector down instead would also avoid the error, but float16 resolves a stride-32 distance of up to 512 px only to ~0.5 px. Regression tests run the full postprocess path on float16 outputs and require the float32 result; both fail before this change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

parkjinman98 · 2026-09-28T07:21:58Z

@mobilint-review Please review this PR again. Here is a guide, so this pass doesn't cover settled ground.

Resolved since the last review

[P2] Handle float16 distribution outputs: fixed in a5178df. Both new decoders upcast their raw heads to float32 before decoding (DAMOYOLODetectionPost._levels, YOLOXDetectionPost._raw_rows). The fix also covers YOLOX's silent counterpart: exp() of a float16 size regression overflows. Regression tests: test_damoyolo_decodes_float16_heads_like_float32 and test_yolox_decodes_float16_output_without_overflowing_exp. Please don't re-raise this. Upcasting instead of casting bin_values down is intentional, for precision.

Intentional decisions, measured or checked against upstream. Please don't flag these as defects.

NMS candidate sort stays unstable. It is argsort(descending=True), not stable, to reproduce Ultralytics' tie order. model-ops' stable sort shifted MXQ mAP only through tied scores (YOLOv8m-pose −0.0038). This was the maintainer's choice.
non_max_suppression keeps a NaN IoU. It uses ~(ratio > thr), which is torchvision semantics. Two zero-area boxes give 0/0 and must not suppress each other.
ImageNet ties favour the higher class index. Top-1 and top-5 come from one stable ranking, which is model-ops' rule. It moves ResNet50 top-1 by +0.0005.
Decode-true pose visibility is sigmoided. The two test_wrapper tests that expected raw values were stale; test_postprocess_hierarchy pins the sigmoid.
YOLOX input is BGR. Upstream uses cv2.imread with no conversion and ValTransform(legacy=False). Feeding RGB measured −4.9 mAP50-95.
DAMO-YOLO input is RGB, letterboxed top-left with zeros. This is the geometry of the December 2022 checkpoints, not upstream HEAD's stretch (39.72 vs 41.79 mAP). keep_ratio is omitted because LetterBox always preserves aspect ratio.
Neither family has a Normalize step. Both take the unscaled 0-255 image, and the ONNX path casts it to the graph's float dtype.
DAMO-YOLO reg_max: 16 means 17 bins. Neither decoder adds a half-cell grid offset; both formulas match upstream.
The new models set file_cfg.local_artifact_only: true. No Hub repository exists yet, so they need a local model_path.
CLAUDE.md and .agents/skills/* are symlinks. They point to AGENTS.md and .claude/skills/*, mirroring mblt-model-ops.

Worth your attention

Layout inference in damoyolo_post._levels. The 640-input stride-8 class map is 80×80×80, so the layout is resolved jointly from the distribution maps.
PostBase.ratio_pads_for and Results._shape_ratio_pad. These are the top-left fallbacks; centered models must pass through unchanged.
The ONNX dtype cast in MBLT_Engine._prepare_onnx_inputs.

Validation: 814 offline tests pass. YOLOX-s scores 40.679 and DAMO-YOLO-T 41.789 mAP50-95 on the full COCO val2017 split (ONNX). YOLOv8m, YOLOv8m-pose and YOLOv8m-seg MXQ predictions are byte-identical to main. There is no MXQ artifact for YOLOX or DAMO-YOLO yet.

github-actions

Codex review

Requested by @parkjinman98.

Verdict

The decoder, layout inference, and top-left geometry changes look consistent with the stated contracts, but the ONNX input conversion still leaves some declared dtype mismatches unconverted.

Suggested next steps

Cast every supported numeric input to the declared ONNX dtype whenever it differs.
Add a regression covering float32 preprocessing output with a tensor(float16) ONNX input.

Trigger: issue_comment
Source: #22 (comment)

Review of #22 found that the ONNX input cast only converted integer and float64 inputs. Normalize's float32 output fed to a graph declaring tensor(float16) or tensor(double) still reached ONNX Runtime unconverted, and ORT rejects the mismatch instead of converting. Once the graph declares a float element type, any integer or floating input of another dtype is now cast to it. A graph declaring no float type keeps the old rule (only float64 narrows to float32), so byte inputs to a uint8 graph pass through. The cast test is parametrized over bytes, float32->float16, float32->double, float64->float, float32->float and uint8->uint8; the float16 and double cases fail before this change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

parkjinman98 · 2026-09-28T08:09:22Z

@mobilint-review Please review this PR again. Both findings from earlier passes are fixed and their threads resolved; this guide covers what's settled.

Resolved. Please don't re-raise these.

[P2] float16 DAMO-YOLO distribution outputs, fixed in a5178df. Both new decoders upcast their raw heads to float32 before decoding. This also covers YOLOX's exp() overflow on float16. Upcasting instead of casting bin_values down is deliberate, for precision.
[P2] Cast every ONNX input dtype mismatch, fixed in c3f19e1. When the graph declares a float type (tensor(float|float16|double)), any integer or floating input of another dtype is cast to it. A graph that declares no float type keeps the old rule (float64 narrows to float32; everything else passes through). That's deliberate: we don't guess conversions for integer-typed graphs.

Intentional decisions, measured or checked against upstream. Please don't flag these as defects.

NMS candidate sort stays unstable (argsort(descending=True)), to match Ultralytics' tie order. This was the maintainer's decision.
non_max_suppression keeps a NaN IoU (torchvision semantics), so zero-area boxes don't suppress each other.
ImageNet ties go to the higher class index via one stable ranking, which is model-ops' rule.
Decode-true pose visibility is sigmoided.
YOLOX input is BGR (verified against upstream; RGB costs 4.9 mAP).
DAMO-YOLO input is RGB, letterboxed top-left with zeros (the December 2022 checkpoints). keep_ratio is omitted because LetterBox always preserves aspect ratio.
Neither family has a Normalize step.
DAMO-YOLO reg_max: 16 means 17 bins, and neither decoder adds a half-cell offset.
The new YAMLs set local_artifact_only: true, because no Hub repository exists yet.
CLAUDE.md and .agents/skills/* are symlinks, following mblt-model-ops' layout.

If you look further, the least-reviewed code is the top-left fallback geometry (PostBase.ratio_pads_for, Results._shape_ratio_pad), which must leave centered models byte-identical, and damoyolo_post._levels' joint layout inference.

Validation: 819 offline tests pass. YOLOX-s scores 40.679 and DAMO-YOLO-T 41.789 mAP50-95 on the full COCO val2017 split (ONNX). Existing MXQ detection, pose and segmentation predictions are byte-identical to main.

github-actions

Codex review

Requested by @parkjinman98.

Verdict

One separate P2 remains. The previously resolved findings and intentional decisions were not re-raised.

Suggested next steps

Make ratio_pads_for batch-aware.
Add a top-left YOLOX/DAMO test with multiple NMS outputs and a shared image-shape tuple.

Trigger: issue_comment
Source: #22 (comment)

Review of #22 found that PostBase.ratio_pads_for built one pad per image *shape* rather than per NMS output. A batched top-left model (YOLOX, DAMO-YOLO) converted with one shared (H, W) got a single derived pad, and the shared conversion helpers rejected it: "Expected 2 ratio_pad values, got 1". Per-image pads over a shared shape failed the other way round. Centered models never reach this branch. A shared shape with shared or no metadata now yields one shared pad, which the helpers broadcast across the batch exactly as they broadcast None. Per-image metadata broadcasts the shared shape to its own length, and per-image shapes stay per image. Regressions run a two-image YOLOX batch through nmsout2eval over all four shape/metadata combinations, and check per-image scaling; the three shared-shape cases fail before this change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

parkjinman98 · 2026-09-28T08:30:20Z

@mobilint-review Please review this PR again. All three findings from earlier passes are fixed, with regression tests that fail before each fix, and their threads are resolved. If nothing new remains, please approve.

Resolved. Please don't re-raise these.

float16 DAMO-YOLO distribution outputs, fixed in a5178df. Both decoders upcast raw heads to float32, which also covers YOLOX's exp() overflow on float16.
ONNX input dtype mismatch, fixed in c3f19e1. Any numeric input is cast to a declared float graph type. Graphs that declare no float type keep the old rule on purpose.
Batched top-left ratio pads, fixed in 88424a3. PostBase.ratio_pads_for keeps the caller's cardinality:
- A shared shape with shared or no metadata gives one shared pad, which the helpers broadcast across the batch as they broadcast None.
- Per-image metadata broadcasts the shape to its own length.
- Per-image shapes stay per image.
All four combinations are covered with a two-image YOLOX batch.

Intentional decisions, measured or checked against upstream. Please don't flag these as defects.

The NMS candidate sort stays unstable, matching Ultralytics' tie order. This was the maintainer's decision.
non_max_suppression keeps a NaN IoU (torchvision semantics).
ImageNet ties go to the higher class index.
Decode-true pose visibility is sigmoided.
YOLOX input is BGR; DAMO-YOLO is RGB, letterboxed top-left with zeros (the December 2022 checkpoints).
Neither family has a Normalize step, and keep_ratio is omitted from the YAMLs.
DAMO-YOLO reg_max: 16 means 17 bins, and neither decoder adds a half-cell offset.
The new YAMLs set local_artifact_only: true.
CLAUDE.md and .agents/skills/* are symlinks.

Scope note: centered (Ultralytics) models never enter the top-left branches. ratio_pads_for returns their metadata untouched, and Results._shape_ratio_pad returns None for them, so their behaviour is byte-identical to main. Existing MXQ detection, pose and segmentation predictions were verified identical.

Validation: 824 offline tests pass. YOLOX-s scores 40.679 and DAMO-YOLO-T 41.789 mAP50-95 on the full COCO val2017 split (ONNX).

github-actions

Codex review

Requested by @parkjinman98.

Verdict

I found one additional P2 issue in asymmetric top-left mask restoration. The three previously resolved findings and intentional decisions are not re-raised.

Suggested next steps

Update mask restoration to crop the resized-image region using asymmetric letterbox geometry.
Add plotting and evaluation coverage for a non-square top-left instance-segmentation input.

Trigger: issue_comment
Source: #22 (comment)

Review of #22 found that scale_masks crops `pad` from both sides of every mask, which only holds for a centered letterbox. With a top-left one (LetterBox.center: false) the padding is all at the bottom and right, so a 640x640 mask from a 480x640 source kept its 160 padding rows and was squeezed into the image: 75% of a full-image mask survived. scale_masks gains `center` (default True). When it is False the crop runs from the top-left pad to pad + round(size * gain), the resized extent. nmsout2eval_seg takes the model's anchoring, the postprocessor passes its letterbox_center, and Results plotting passes letterbox_center(pre_cfg). The centered formula is untouched, so Ultralytics models stay byte- identical (asserted with atol=rtol=0 over three pad geometries). Also found alongside: nmsout2eval_seg normalized a shared image shape without the batch size and raised "Expected 2 image shapes, got 1" for a two-image batch, unlike the detection conversion. It now broadcasts the shape to the NMS batch; calls that already worked return the same values. Regressions cover the top-left crop, centered invariance, non-square top-left eval (mask area) and plotting (interior pixel tint), and the shared-shape segmentation batch; all but the invariance check fail before this change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

An audit of the remaining padding arithmetic after the top-left mask fix found one more shape-only fallback: DepthPost and SemanticSegPost fill a missing ratio_pad through resolve_ratio_pads, which always derived a centered letterbox. A top-left dense model restored without recorded metadata was cropped at rows 80-560 of a 640 map from a 480x640 source, pulling 80 padding rows into the image. resolve_ratio_pads takes `center`, and both dense postprocessors pass letterbox_center(pre_cfg). The default stays centered, so every current model is unchanged. The other padding paths were checked and already hold for any anchoring: scale_boxes/coords/rboxes subtract the recorded pad, LetterBoxGeometry.crop_bounds spans pad..pad+resized, the prototype ROI crop has no letterbox, and the evaluators always pass recorded pads. Regression: a top-left depth map with no metadata restores exactly its image rows; it fails before this change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

parkjinman98 · 2026-09-28T08:56:07Z

@mobilint-review Please review this PR again. All four earlier findings are fixed, each with regression tests that fail before the fix, and every thread is resolved. If nothing new remains, please approve.

Resolved. Please don't re-raise these.

float16 decoding. Fixed in a5178df: the raw YOLOX and DAMO-YOLO heads are upcast to float32.
ONNX input dtype. Fixed in c3f19e1: any numeric input is cast to the float type the graph declares. Graphs that declare no float type deliberately keep the old rule.
Batched top-left ratio pads. Fixed in 88424a3: ratio_pads_for keeps the caller's cardinality. A shared shape gives one shared pad, which is broadcast.
Top-left mask crop. Fixed in c97afea: scale_masks(center=False) crops from the pad to the resized extent. nmsout2eval_seg and Results plotting pass the model's anchoring, and centered models are byte-identical (asserted with atol=rtol=0). The same commit fixes nmsout2eval_seg rejecting one shared shape for a batch.

Proactive fix, 48d8ae1. Findings 3 and 4 were both centered-letterbox assumptions, so I audited every remaining padding path. One more fallback derived centered geometry: resolve_ratio_pads, used by DepthPost and SemanticSegPost when no ratio_pad is recorded. It now takes the model's letterbox_center, with a regression test.

The other padding paths already work for any anchoring, so please treat them as checked:

scale_boxes, scale_coords and scale_rboxes subtract the recorded pad.
LetterBoxGeometry.crop_bounds spans from the pad to the pad plus the resized extent.
_mask_scale_crop_bounds and process_mask_upsample upsample prototypes to the input's own aspect ratio, so there is no letterbox crop.
The COCO, WiderFace and DOTA evaluators always pass recorded pads from preprocessing.

Intentional decisions. Please don't flag these as defects.

Unstable NMS candidate sort, to match Ultralytics' tie order (maintainer's decision).
non_max_suppression keeps a NaN IoU (torchvision semantics).
ImageNet ties go to the higher class index.
Decode-true pose visibility is sigmoided.
YOLOX input is BGR; DAMO-YOLO is RGB, letterboxed top-left with zeros.
Neither family has a Normalize step.
DAMO-YOLO reg_max: 16 means 17 bins, with no half-cell offset.
The new YAMLs set local_artifact_only: true.
CLAUDE.md and .agents/skills/* are symlinks.
No shipped top-left model is segmentation, pose or dense. The top-left mask and dense fixes are correctness for configurations that set LetterBox.center: false; YOLOX and DAMO-YOLO are detection only, and post_cfg.head is rejected on other tasks.

Validation: 830 offline tests pass. YOLOX-s scores 40.679 and DAMO-YOLO-T 41.789 mAP50-95 on the full COCO val2017 split (ONNX). Existing MXQ detection, pose and segmentation predictions are byte-identical to main.

github-actions

Codex review

Requested by @parkjinman98.

Verdict

One new P2 remains: semantic target letterboxing does not honor top-left anchoring. The YOLOX/DAMO-YOLO changes and listed resolved findings otherwise look consistent.

Suggested next steps

Propagate the model's center setting through semantic target preprocessing.
Add a non-square semantic evaluation test for center=false.

Trigger: issue_comment
Source: #22 (comment)

Review of #22 found that letterbox_semantic_mask always centered the target. With LetterBox.center: false the ADE20K and Cityscapes loaders got top-left image metadata and a centered target, and their geometry check failed on every non-square image. letterbox_semantic_mask, get_ade20k_loader and get_cityscapes_loader take `center` (default True), and eval_semantic_segmentation passes the model's letterbox_center(pre_cfg). A repo-wide sweep of letterbox geometry for the same bug found one more shape-only fallback: eval_dota derived a missing ratio_pad as centered. It now takes the model's anchoring too. The remaining call sites either receive explicit pads (the scale_* helpers, whose documented default is centered) or use only the resized extent, which does not depend on it. Regressions: both semantic loaders on a non-square top-left sample (the padding lands at the bottom as ignore_label), the evaluator passing center to its loader, and the DOTA fallback pad. The DOTA test fake gains the pre_cfg every real engine has. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

parkjinman98 · 2026-09-28T09:22:51Z

@mobilint-review Please review this PR again. All five earlier findings are fixed, each with regression tests that fail before the fix, and every thread is resolved. If nothing new remains, please approve.

Resolved. Please don't re-raise these.

float16 decoding (a5178df).
ONNX input dtype (c3f19e1).
Batched top-left ratio pads (88424a3).
Top-left mask crop, plus a shared-shape seg batch (c97afea).
Top-left semantic targets (914a7a0): letterbox_semantic_mask, the ADE20K and Cityscapes loaders, and eval_semantic_segmentation carry the model's center.

Proactive fixes: the dense resolve_ratio_pads fallback (48d8ae1) and the DOTA ground-truth fallback (914a7a0).

The top-left surface is now fully inventoried. Findings 3-5 were all one class of bug, so I swept every letterbox-geometry call site in mblt_vision/, compile/ and benchmark/. Each one either follows the model's anchoring or doesn't need it:

Call site	Status
`LetterBox.with_ratio_pad` (`preprocess/letterbox.py`)	Source of truth; honours `center`/`padding_value`.
`letterbox_semantic_mask` (both semantic loaders)	Takes the model's `center`.
`PostBase.ratio_pads_for` (detection/face/seg/pose/OBB eval wrappers)	Top-left fallback, batch-aware.
`resolve_ratio_pads` (`DepthPost`, `SemanticSegPost`)	Takes the model's `center`.
`eval_dota._ratio_pad_for_shape`	Takes the model's `center`.
`Results._shape_ratio_pad`, `_restore_dense_map`, `scale_masks(center=)` (plotting)	Follow `letterbox_center(pre_cfg)`.
`scale_boxes` / `scale_coords` / `scale_rboxes` / `clip`	Documented centered default for `ratio_pad=None`. Every model-aware caller passes explicit pads.
`crop_letterbox`, `LetterBoxGeometry.crop_bounds`	Use explicit pads; the resized extent is independent of anchoring.
`process_mask_upsample`, `_mask_scale_crop_bounds`	Prototype→input upsampling at the same aspect ratio; no letterbox.

Intentional decisions. Please don't flag these as defects.

Unstable NMS candidate sort, to match Ultralytics' tie order (maintainer's decision).
non_max_suppression keeps a NaN IoU.
ImageNet ties go to the higher class index.
Decode-true pose visibility is sigmoided.
YOLOX input is BGR; DAMO-YOLO is RGB, letterboxed top-left with zeros.
Neither family has a Normalize step.
DAMO-YOLO reg_max: 16 means 17 bins, with no half-cell offset.
The new YAMLs set local_artifact_only: true.
CLAUDE.md and .agents/skills/* are symlinks.
Centered defaults. Every new center parameter defaults to True, so existing callers and all shipped non-YOLOX/DAMO models are unchanged.
No shipped top-left model is segmentation, pose, OBB or dense. Those fixes are correctness for configs that set LetterBox.center: false.

Validation: 834 offline tests pass. YOLOX-s scores 40.679 and DAMO-YOLO-T 41.789 mAP50-95 on the full COCO val2017 split (ONNX). Existing MXQ detection, pose and segmentation predictions are byte-identical to main.

github-actions

Codex review

Requested by @parkjinman98.

Verdict

Found one new P2 geometry defect. The five prior findings remain resolved and are not re-raised.

Suggested next steps

Fix the center=False fallback when ratio_pad is absent.
Add a regression test for scale_masks(..., center=False) without explicit metadata.

Trigger: issue_comment
Source: #22 (comment)

Review of #22 found a gap in c97afea: when scale_masks derives the padding itself (ratio_pad=None), it still halved it before computing top/left, and the center=False branch only corrected bottom/right. A 480x640 image letterboxed to 640x640 was cropped at rows 80:560 instead of 0:480, keeping 83% of a full-image mask. With no metadata and a top-left letterbox the leading padding is now zero, since all of it sits at the bottom and right. The centered branch is unchanged. Regression: for landscape and portrait sources, the no-metadata top-left crop keeps every image row and equals the explicit-metadata result exactly; both fail before this change. The other top-left fallbacks derive their pads through resolve_ratio_pad/from_shapes, which already return zero, and are covered by their own tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

parkjinman98 · 2026-09-28T09:46:30Z

@mobilint-review Please review this PR again. All six earlier findings are fixed, each with regression tests that fail before the fix, and every thread is resolved. If nothing new remains, please approve.

Resolved. Please don't re-raise these.

float16 decoding (a5178df).
ONNX input dtype (c3f19e1).
Batched top-left ratio pads (88424a3).
Top-left mask crop with explicit metadata (c97afea).
Top-left semantic targets (914a7a0).
Top-left mask crop without metadata (7f93ce8). scale_masks(ratio_pad=None, center=False) now starts at the origin. For landscape and portrait sources, the test asserts it equals the explicit-metadata result exactly.

Proactive fixes: the dense resolve_ratio_pads fallback (48d8ae1) and the DOTA ground-truth fallback (914a7a0).

Top-left geometry invariant. For every top-left entry point, the derived-metadata path must equal the explicit ((r, r), (0, 0)) path. scale_masks was the only function that derived padding internally. Every other fallback goes through resolve_ratio_pad → LetterBoxGeometry.from_shapes(center=False), which returns pad == (0, 0). The call-site inventory from my previous comment still holds.

Intentional decisions. Please don't flag these as defects.

Unstable NMS candidate sort, to match Ultralytics' tie order (maintainer's decision).
non_max_suppression keeps a NaN IoU.
ImageNet ties go to the higher class index.
Decode-true pose visibility is sigmoided.
YOLOX input is BGR; DAMO-YOLO is RGB, letterboxed top-left with zeros.
Neither family has a Normalize step.
DAMO-YOLO reg_max: 16 means 17 bins, with no half-cell offset.
The new YAMLs set local_artifact_only: true.
CLAUDE.md and .agents/skills/* are symlinks.
Centered defaults. Every new center parameter defaults to True, so existing callers and shipped non-YOLOX/DAMO models are unchanged.
No shipped top-left model is segmentation, pose, OBB or dense.

Validation: 836 offline tests pass. YOLOX-s scores 40.679 and DAMO-YOLO-T 41.789 mAP50-95 on the full COCO val2017 split (ONNX). Existing MXQ detection, pose and segmentation predictions are byte-identical to main.

#25) * Rank NMS and end-to-end candidates in Ultralytics' validation order Two divergences from Ultralytics 8.4.137, found while aligning mblt-model-ops with it (mblt-model-ops !230): - Tie order. AGENTS.md required the unstable argsort(descending=True), on the belief that it reproduces Ultralytics. It does not: non_max_suppression passes up to 30000 candidates straight to torchvision.ops.nms, whose sort is stable on CPU and CUDA; the argsort above that cap and an end-to-end head's torch.topk (k = 300) are stable on CUDA, where Ultralytics validates. On CPU, tied scores came out in another order, and quantized MXQ scores tie often. Every candidate ranking now goes through common.descending_order, a stable descending sort, which gives Ultralytics' CUDA order on every device. The #22 measurement (YOLOv8m -0.00047 etc.) compared two tie orders, not either one against Ultralytics. - dual_topk's row cap. It ranked only the anchors above the threshold and then capped its second stage at that anchor count, so whenever fewer than max_det anchors cleared the threshold every further class of those anchors was dropped: 100 anchors with three classes each kept 100 rows where Detect.get_topk_index keeps 300. The second stage now keeps up to max_det (anchor, class) pairs. Verified against Ultralytics 8.4.137 run on CUDA, with int8-quantized, tie-heavy inputs, 6 trials per case: dual_topk (sparse multi-class, dense) and the multi-label NMS candidate path are identical on CPU and CUDA tensors (24/24). Before this change every CPU case differed, and sparse multi-class dual_topk differed on CUDA too. tests/test_ultralytics_selection_order.py pins both. Run against the previous code, it fails on the CPU tie order of NMS and dual_topk and on the row cap on both devices. AGENTS.md and the mblt-vision skill state the new rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Score DOTAv1 the way Ultralytics validates and publishes The 15 OBB models are Ultralytics checkpoints, so their own repository is the reference, as mblt-model-ops now holds too (mblt-model-ops !230): - Difficult objects (flag 1 or 2) load as ordinary targets. Ultralytics' convert_dota_to_yolo_obb keeps the eight coordinates and the class name and drops the flag, so its validation counts them. The loader used to route them into ignore regions, the DOTA devkit protocol behind the test server. evaluate_dota_predictions still honours ignore regions a caller passes explicitly; the loader produces none. The flag is still validated. - Rotated mAP50 is the primary metric, the number Ultralytics publishes for its OBB models (mAP test 50); mAP50-95 becomes secondary. DOTAResult, `mblt-vision val`'s print order and the benchmark runner's score_name follow. Tests: both loader paths (normalized and official val_original labels) count a difficult object; the supplied-ignore-region test now says what it covers; the benchmark runner reports map50. AGENTS.md, the mblt-vision skill and mblt_vision/README.md state the rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Run YOLOv7 at the IoU threshold its repository uses; bump to 0.0.7 WongKinYiu/yolov7's test.py defaults to --iou-thres 0.65, and its README reproduces every published COCO number with `--iou 0.65`; the iou_thres=0.6 in test()'s signature is always overridden by the command line. All six YOLOv7 models (YOLOv7, -x, -w6, -e6, -d6, -e6e) used 0.6. mblt-model-ops makes the same change (mblt-model-ops !230). The README's YOLOv7 scores were measured at 0.6 and are marked for re-measurement. __version__ moves to 0.0.7: this release changes NMS ordering, end-to-end selection and DOTAv1 scoring, so mblt-model-ops' parity pin can follow it. AGENTS.md and the mblt-vision skill record the YOLOv7 threshold. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

parkjinman98 and others added 2 commits September 28, 2026 15:31