Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ faiss_indexes/

# Test images and videos (use external paths or download separately).
#
# F-29/F-71 (fresh-start E2E findings 2026-09-25): yolo-api bind-mounts
# F-29/F-71 (fresh-start E2E findings 2026-09-25): api bind-mounts
# ./test_images. On a fresh clone that host dir doesn't exist, so Docker
# auto-creates it root-owned on the first `up`, and `make
# download-test-images` (running as the host user) can't write into it.
Expand Down
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,15 @@ history of this codebase and was never published. This release is `[0.4.1]`.

### Changed

- **Breaking:** the compose service `yolo-api` is renamed `api` (#62). `docker compose exec yolo-api ...`,
`docker compose logs yolo-api` and `./openprocessor logs|restart yolo-api` become
`... api`; the Prometheus job and Loki service label are `api` too. Custom overrides, scripts
and dashboards that name the old service must change. The container name
(`<project>-api`), the `op-api` network alias and all data volumes are unchanged. An existing
install is upgraded by `./openprocessor upgrade` (or `docker compose up -d --remove-orphans`),
which replaces the old container; every start path (`start`, `make up`, `scripts/setup.sh`,
the installer's control-plane-only path) now passes `--remove-orphans`, because the old
container holds the same name and would otherwise make the new one fail to create.
- The segmenter image moves to PyTorch 2.14.1 (`pytorch/pytorch:2.14.1-cuda12.6-cudnn9-runtime`,
superseding dependabot #147) and `alpine/git` v2.54.0. The new base is Ubuntu 24.04, so the
image now runs Python 3.12 (was 3.11), installs with `PIP_BREAK_SYSTEM_PACKAGES=1` and
Expand Down
6 changes: 3 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ Docker socket (read its header first).
| Service | Profile | Port | Role |
|---|---|---|---|
| `triton-server` | none | 4600 HTTP, 4601 gRPC, 4602 metrics | TensorRT models, explicit model control |
| `yolo-api` | none | 4603 | FastAPI, all routes |
| `api` | none | 4603 | FastAPI, all routes |
| `opensearch` | none | 4607 | k-NN store and the curation datastore |
| `curation-detection-worker`, `curation-vlm-worker`, `curation-auto-label-worker`, `curation-cluster-refresh`, `curation-evaluator` | `curation` | | Region cascade, VLM loop, auto-label driver, clustering refresh, bake-off runner |
| `segmenter` | `segmenter` | 4611 | Region proposals from a text prompt |
Expand Down Expand Up @@ -121,8 +121,8 @@ The management CLI works in a checkout (`./openprocessor`, or the shim
`bench` and `setup` need a checkout. Details:
[INSTALLATION.md](INSTALLATION.md#the-openprocessor-cli).

Python source is mounted into `yolo-api` in a checkout, so most edits are live;
restart `yolo-api` after route changes. Rebuild images only when a Dockerfile or
Python source is mounted into `api` in a checkout, so most edits are live;
restart `api` after route changes. Rebuild images only when a Dockerfile or
`requirements.txt` changes.

## Core API shape
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ Pre-commit rejects a commit that leaves a contract stale. See

The api and the workers bind-mount the source. Python modules already imported by a
running container go stale, which shows up as transient 500 `ImportError`s. After a pull
or merge run `make dev-restart` (restarts `yolo-api` and the detection, VLM, auto-label
or merge run `make dev-restart` (restarts `api` and the detection, VLM, auto-label
and cluster-refresh workers without recreating them). Restart does not re-read `.env`:
after editing it run `make dev-up` instead (plain `docker restart` or `make dev-restart`
keep the old values).
Expand Down
6 changes: 3 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# OpenProcessor FastAPI Service
# Multi-stage build with non-root user and production defaults
# =============================================================================
# Builds the yolo-api FastAPI service for visual AI inference.
# Builds the api FastAPI service for visual AI inference.
# docker-compose.yml overrides CMD with environment-specific worker counts.
#
# VOLUME MOUNTS (for development):
Expand Down Expand Up @@ -85,7 +85,7 @@ LABEL org.opencontainers.image.revision=${OP_BUILD_SHA}
# Debian point-release security fixes.
# procps -> pgrep, used by the curation worker services' healthchecks
# (docker-compose.yml `profiles: [curation]`) — this image is shared
# between yolo-api and those long-running worker processes.
# between api and those long-running worker processes.
RUN apt-get update && apt-get upgrade -y \
&& apt-get install -y --no-install-recommends \
curl \
Expand Down Expand Up @@ -163,7 +163,7 @@ EXPOSE 8000

# /live = process liveness only. /health now probes dependencies (Triton,
# OpenSearch) and returns 503 while any is down — gating the container's
# health on it would mark yolo-api unhealthy (and cascade via
# health on it would mark api unhealthy (and cascade via
# depends_on: service_healthy) whenever a *downstream* dep degrades.
HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
CMD curl -f http://localhost:8000/live || exit 1
Expand Down
2 changes: 1 addition & 1 deletion INSTALLATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -522,7 +522,7 @@ Or generate automatically:
### 5. Start Containers (for export)

```bash
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d triton-server yolo-api
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d triton-server api
```

`make up` and `./openprocessor start` add the dev overlay for you and start
Expand Down
12 changes: 6 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ V := .venv/bin
# Prefer the project venv's interpreter when it exists; fall back to
# system python3 for a fresh checkout that hasn't created .venv yet.
PYTHON ?= $(if $(wildcard .venv/bin/python),.venv/bin/python,python3)
API_SERVICE := yolo-api
API_SERVICE := api
AUTO_LABEL_SERVICE := curation-auto-label-worker
TRITON_SERVICE := triton-server
OPENSEARCH_SERVICE := opensearch
Expand Down Expand Up @@ -96,7 +96,7 @@ ensure-host-bind-mount-dirs: ## F-29/F-71: pre-create bind-mount source dirs as
.PHONY: up
up: ensure-host-bind-mount-dirs ## Start the core stack (Triton + API + OpenSearch) -- see 'make up-monitoring' for Prometheus/Grafana/Loki/dcgm
@echo "Starting core services..."
$(COMPOSE) up -d
$(COMPOSE) up -d --remove-orphans
@echo ""
@echo "Services starting. Check status with: make status"
@echo "API available at: http://localhost:$(API_PORT)"
Expand All @@ -115,27 +115,27 @@ cluster-cpu: ## Recreate the auto-label worker without GPU passthrough (sklearn
.PHONY: up-monitoring
up-monitoring: ensure-host-bind-mount-dirs ## Start the core stack PLUS monitoring (F-3: opt-in on a shared host -- alloy mounts docker.sock and tails every container, dcgm-exporter reserves all GPUs)
@echo "Starting core + monitoring services..."
$(COMPOSE) --profile monitoring up -d
$(COMPOSE) --profile monitoring up -d --remove-orphans
@echo ""
@echo "Services starting. Check status with: make status"
@echo "API available at: http://localhost:$(API_PORT)"
@echo "Grafana dashboard: http://localhost:$(GRAFANA_PORT) (admin/admin)"

.PHONY: dev-up
dev-up: ensure-host-bind-mount-dirs ## Single canonical dev deployment: dev overlay + GPU arbiter overlay + curation/segmenter/vlm/training profiles
$(DEV_COMPOSE) up -d --build
$(DEV_COMPOSE) up -d --build --remove-orphans

.PHONY: dev-monitoring-up
dev-monitoring-up: ensure-host-bind-mount-dirs ## 'make dev-up' PLUS the monitoring profile (Prometheus/Grafana/Loki/Alloy/DCGM + Dashboards); opt-in as in up-monitoring
$(DEV_COMPOSE) --profile monitoring up -d --build
$(DEV_COMPOSE) --profile monitoring up -d --build --remove-orphans

.PHONY: dev-down
dev-down: ## Stop the dev deployment started by 'make dev-up' (keeps volumes)
$(DEV_COMPOSE) down

.PHONY: dev-restart
dev-restart: ## Restart the code-mounting containers (api + workers) after pulling/merging code; does not recreate or re-read .env
$(DEV_COMPOSE) restart yolo-api curation-detection-worker curation-vlm-worker curation-auto-label-worker curation-cluster-refresh
$(DEV_COMPOSE) restart api curation-detection-worker curation-vlm-worker curation-auto-label-worker curation-cluster-refresh

.PHONY: dev-ps
dev-ps: ## Show the dev deployment's containers
Expand Down
24 changes: 12 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ address also narrows Cropwright to that interface. See
```bash
cd openprocessor
./openprocessor status # services and health
./openprocessor logs yolo-api -f # live logs
./openprocessor logs api -f # live logs
./openprocessor sample coco # fetch a public COCO sample (200 images)
./openprocessor vlm list # the local VLM catalog
./openprocessor upgrade # to the latest release (backs up first)
Expand Down Expand Up @@ -271,14 +271,14 @@ Government Work"; never NonCommercial or NoDerivs). Then:
# (PUT /curation/projects/{project}/ingest/policy); the old
# OP_INGEST_PRIMARY_CLASS_IDS env var is retired and ignored.
# OP_INGEST_PRIMARY_DETECTOR_MODEL picks the detector; to use another
# model set it in .env and recreate yolo-api (see docs/CURATION.md).
# model set it in .env and recreate api (see docs/CURATION.md).
# 3. Point OP_SOURCE_ROOT_HOST at data/samples in .env (the compose mount
# target is fixed at /data/source):
echo 'OP_SOURCE_ROOT_HOST=./data/samples' >> .env
docker compose up -d --force-recreate yolo-api
docker compose up -d --force-recreate api
# 4. --root is a container path under the /data/source mount, not a
# host-relative one:
docker compose exec yolo-api python scripts/curation/ingest_walker.py \
docker compose exec api python scripts/curation/ingest_walker.py \
--root /data/source/coco_va_readme/images --project cars \
--api-base http://localhost:8000/curation
```
Expand All @@ -291,7 +291,7 @@ to stop (volumes are kept). Set `COMPOSE_PROJECT_NAME`, `OP_IMAGE_REPO` and
published ones.

In a source checkout use `make up` and `make` targets, which add the dev
overlay; in an installed directory run `./openprocessor restart yolo-api` after
overlay; in an installed directory run `./openprocessor restart api` after
editing `.env`. `make sample-coco` (800 images plus side sets),
`make sample-plates` (300 Open Images V7 images) and `make sample-clean` use
the same tool. See
Expand Down Expand Up @@ -432,7 +432,7 @@ YOLO26.
(`export/export_yolo26.py`, natively NMS-free engines):

```bash
docker compose exec yolo-api python /app/export/export_yolo26.py --models small
docker compose exec api python /app/export/export_yolo26.py --models small
curl -X POST http://localhost:4603/models/yolo26_small_trt/load
```

Expand Down Expand Up @@ -750,7 +750,7 @@ Client (Port 4603)
|
v
+----------+ +-----------------------------+
| yolo-api |---->| curation workers (profile) |
| api |---->| curation workers (profile) |
+----------+ | detection, VLM, auto-label, |
| | cluster refresh, evaluator |
v +-----------------------------+
Expand All @@ -762,7 +762,7 @@ Client (Port 4603)

**Services** (compose profile in brackets):

- `yolo-api` (4603): FastAPI service handling all requests.
- `api` (4603): FastAPI service handling all requests.
- `triton-server` (4600-4602): Triton with TensorRT models.
- `opensearch` (4607): vector database and the curation datastore.
- `curation-detection-worker`, `curation-vlm-worker`,
Expand Down Expand Up @@ -864,15 +864,15 @@ running stack over HTTP, so they read their target ports from the environment
.venv/bin/python tests/validate_visual_results.py 2>&1 | tee test_results/visual_validation.txt
```

**Docker-only path.** The production `yolo-api` image installs only
**Docker-only path.** The production `api` image installs only
`requirements.txt` (no `pytest`, no `requirements-test.txt`), so a bare
`docker compose exec yolo-api pytest ...` fails with `executable file not
`docker compose exec api pytest ...` fails with `executable file not
found`. Install the test deps into the running container first (not persisted
across a recreate):

```bash
docker compose exec yolo-api pip install -r requirements-test.txt
docker compose exec yolo-api python -m pytest tests/ -q --ignore=tests/live
docker compose exec api pip install -r requirements-test.txt
docker compose exec api python -m pytest tests/ -q --ignore=tests/live
```

See [CONTRIBUTING.md](CONTRIBUTING.md) for the live write-path harness and the
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ For custom benchmarks with detailed metrics:

```bash
# Run inside container
docker compose exec yolo-api python /app/benchmarks/scripts/benchmark.py
docker compose exec api python /app/benchmarks/scripts/benchmark.py

# Or use make target
make bench-python
Expand Down
2 changes: 1 addition & 1 deletion docker-compose.dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ services:
context: .
dockerfile: Dockerfile.triton

yolo-api:
api:
build:
context: .
dockerfile: Dockerfile
Expand Down
8 changes: 4 additions & 4 deletions docker-compose.gpu-arbiter.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Opt-in overlay: mounts the host Docker socket into yolo-api so the GPU
# Opt-in overlay: mounts the host Docker socket into api so the GPU
# arbiter (src/services/training/gpu_arbiter.py) can stop/restart
# GPU-resident sibling containers around a training run
# (OP_GPU_ARBITER_CONTAINERS, OP_GPU_ARBITER_TRAINER_CONTAINER -- see
Expand All @@ -11,10 +11,10 @@
# /var/run/docker.sock into any container gives that container's process
# root-equivalent control over the whole Docker host -- it can start,
# stop or inspect *any* container, not just the ones this project owns,
# and a container-escape bug in yolo-api's dependency stack becomes a
# and a container-escape bug in api's dependency stack becomes a
# host-level compromise. This is why it's a separate, explicitly opt-in
# file instead of a default mount. Only enable it on a host you trust
# yolo-api's dependency stack on, and prefer scoping
# api's dependency stack on, and prefer scoping
# OP_GPU_ARBITER_CONTAINERS to the smallest container list a deployment
# actually needs coordinated.
#
Expand All @@ -27,7 +27,7 @@
# OP_GPU_ARBITER_TRAINER_CONTAINER=<container-name>

services:
yolo-api:
api:
volumes:
# NOT :ro -- stop/start calls over the socket need write access;
# a read-only bind only restricts the mount point, not what the
Expand Down
Loading
Loading