Skip to content

feat(vlm): optional per-pack registry prior for open-vocabulary labeling (#61) - #185

Merged
davidamacey merged 2 commits into
mainfrom
feat/61-registry-prior
Oct 9, 2026
Merged

davidamacey merged 2 commits into
mainfrom
feat/61-registry-prior

Conversation

@davidamacey

Copy link
Copy Markdown
Owner

Refs #61 (item 2). Not closing: the acceptance (overall accuracy on the COCO oracle, prior on vs off) is a live measurement that is still pending.

What

A prompt pack gets registry_prior_top_k (integer, 0 = off by default, hard max 50). When > 0, open-vocabulary VLM labeling (POST /vlm/label_batch and the auto-label pipeline VLM stage) adds to the prompt the top-k registry class names, ranked by validated-item count (ties by name), plus up to k pending new-class proposal names.

Design

  • Built server-side from project state (registry_prior_source.py, aggregations on the bound project's items index, holdout excluded); no client-supplied lists. Names only, never indexes.
  • A hint, not a constraint: reply parsing is unchanged and still accepts any answer.
  • Fail closed: if the pack asks for the prior and the counts cannot be read, label_batch returns 503 and the pipeline returns a stage error, rather than labeling without it.
  • Validation: non-integer or bool is pack_field_missing; outside 0..50 is the new pack_field_out_of_range. Served in the pack schema as kind: "int", group open_classify. Contracts regenerated; frontend PackSchemaField.kind, PackFieldValue, fixture, test pin and CHANGELOG updated (the editor shows an unknown kind read-only and saves it unchanged).
  • Lock rule and test_holdout untouched: only prompt text changes.
  • Not done: a per-item prior from detector_class_name; this is a per-batch registry prior.

Measurement harness (offline-tested only)

scripts/curation/bakeoff/vlm_prior_oracle.py (dump per run, compare off vs on). No live VLM/GPU run was done.

Verified

Items 1 (proposal denylist) and 3 (label_agreement) of the issue are already on main.

…ing (#61)

A prompt pack's registry_prior_top_k (0 = off, max 50) adds the top-k registry
classes by validated count and the pending new-class proposal names to the
open-vocabulary prompt as a hint. Built server-side from project state, ordered
deterministically; replies outside the list are still accepted; the run refuses
(503 / stage error) when the counts cannot be read. Served in the pack schema as
kind int, validated with the new pack_field_out_of_range code. Adds an offline
on/off scorer under scripts/curation/bakeoff.
@davidamacey
davidamacey merged commit d4c3069 into main Oct 9, 2026
12 of 13 checks passed
@davidamacey
davidamacey deleted the feat/61-registry-prior branch October 9, 2026 23:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant