diff --git a/.gitignore b/.gitignore index ce6e0342e..eff436750 100644 --- a/.gitignore +++ b/.gitignore @@ -82,7 +82,7 @@ faiss_indexes/ # Test images and videos (use external paths or download separately). # -# F-29/F-71 (fresh-start E2E findings 2026-09-25): yolo-api bind-mounts +# F-29/F-71 (fresh-start E2E findings 2026-09-25): api bind-mounts # ./test_images. On a fresh clone that host dir doesn't exist, so Docker # auto-creates it root-owned on the first `up`, and `make # download-test-images` (running as the host user) can't write into it. diff --git a/CHANGELOG.md b/CHANGELOG.md index 8c23a23a3..3689e177f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -47,6 +47,15 @@ history of this codebase and was never published. This release is `[0.4.1]`. ### Changed +- **Breaking:** the compose service `yolo-api` is renamed `api` (#62). `docker compose exec yolo-api ...`, + `docker compose logs yolo-api` and `./openprocessor logs|restart yolo-api` become + `... api`; the Prometheus job and Loki service label are `api` too. Custom overrides, scripts + and dashboards that name the old service must change. The container name + (`-api`), the `op-api` network alias and all data volumes are unchanged. An existing + install is upgraded by `./openprocessor upgrade` (or `docker compose up -d --remove-orphans`), + which replaces the old container; every start path (`start`, `make up`, `scripts/setup.sh`, + the installer's control-plane-only path) now passes `--remove-orphans`, because the old + container holds the same name and would otherwise make the new one fail to create. - The segmenter image moves to PyTorch 2.14.1 (`pytorch/pytorch:2.14.1-cuda12.6-cudnn9-runtime`, superseding dependabot #147) and `alpine/git` v2.54.0. The new base is Ubuntu 24.04, so the image now runs Python 3.12 (was 3.11), installs with `PIP_BREAK_SYSTEM_PACKAGES=1` and diff --git a/CLAUDE.md b/CLAUDE.md index 3251b6175..49e42624a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -78,7 +78,7 @@ Docker socket (read its header first). | Service | Profile | Port | Role | |---|---|---|---| | `triton-server` | none | 4600 HTTP, 4601 gRPC, 4602 metrics | TensorRT models, explicit model control | -| `yolo-api` | none | 4603 | FastAPI, all routes | +| `api` | none | 4603 | FastAPI, all routes | | `opensearch` | none | 4607 | k-NN store and the curation datastore | | `curation-detection-worker`, `curation-vlm-worker`, `curation-auto-label-worker`, `curation-cluster-refresh`, `curation-evaluator` | `curation` | | Region cascade, VLM loop, auto-label driver, clustering refresh, bake-off runner | | `segmenter` | `segmenter` | 4611 | Region proposals from a text prompt | @@ -121,8 +121,8 @@ The management CLI works in a checkout (`./openprocessor`, or the shim `bench` and `setup` need a checkout. Details: [INSTALLATION.md](INSTALLATION.md#the-openprocessor-cli). -Python source is mounted into `yolo-api` in a checkout, so most edits are live; -restart `yolo-api` after route changes. Rebuild images only when a Dockerfile or +Python source is mounted into `api` in a checkout, so most edits are live; +restart `api` after route changes. Rebuild images only when a Dockerfile or `requirements.txt` changes. ## Core API shape diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 3d2e6c075..0eb7efb60 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -90,7 +90,7 @@ Pre-commit rejects a commit that leaves a contract stale. See The api and the workers bind-mount the source. Python modules already imported by a running container go stale, which shows up as transient 500 `ImportError`s. After a pull -or merge run `make dev-restart` (restarts `yolo-api` and the detection, VLM, auto-label +or merge run `make dev-restart` (restarts `api` and the detection, VLM, auto-label and cluster-refresh workers without recreating them). Restart does not re-read `.env`: after editing it run `make dev-up` instead (plain `docker restart` or `make dev-restart` keep the old values). diff --git a/Dockerfile b/Dockerfile index 672e9be91..d74a2d6eb 100644 --- a/Dockerfile +++ b/Dockerfile @@ -2,7 +2,7 @@ # OpenProcessor FastAPI Service # Multi-stage build with non-root user and production defaults # ============================================================================= -# Builds the yolo-api FastAPI service for visual AI inference. +# Builds the api FastAPI service for visual AI inference. # docker-compose.yml overrides CMD with environment-specific worker counts. # # VOLUME MOUNTS (for development): @@ -85,7 +85,7 @@ LABEL org.opencontainers.image.revision=${OP_BUILD_SHA} # Debian point-release security fixes. # procps -> pgrep, used by the curation worker services' healthchecks # (docker-compose.yml `profiles: [curation]`) — this image is shared -# between yolo-api and those long-running worker processes. +# between api and those long-running worker processes. RUN apt-get update && apt-get upgrade -y \ && apt-get install -y --no-install-recommends \ curl \ @@ -163,7 +163,7 @@ EXPOSE 8000 # /live = process liveness only. /health now probes dependencies (Triton, # OpenSearch) and returns 503 while any is down — gating the container's -# health on it would mark yolo-api unhealthy (and cascade via +# health on it would mark api unhealthy (and cascade via # depends_on: service_healthy) whenever a *downstream* dep degrades. HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \ CMD curl -f http://localhost:8000/live || exit 1 diff --git a/INSTALLATION.md b/INSTALLATION.md index 882da1bb0..f3791c8e7 100644 --- a/INSTALLATION.md +++ b/INSTALLATION.md @@ -522,7 +522,7 @@ Or generate automatically: ### 5. Start Containers (for export) ```bash -docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d triton-server yolo-api +docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d triton-server api ``` `make up` and `./openprocessor start` add the dev overlay for you and start diff --git a/Makefile b/Makefile index 14a5a10bb..ba9671b8f 100644 --- a/Makefile +++ b/Makefile @@ -21,7 +21,7 @@ V := .venv/bin # Prefer the project venv's interpreter when it exists; fall back to # system python3 for a fresh checkout that hasn't created .venv yet. PYTHON ?= $(if $(wildcard .venv/bin/python),.venv/bin/python,python3) -API_SERVICE := yolo-api +API_SERVICE := api AUTO_LABEL_SERVICE := curation-auto-label-worker TRITON_SERVICE := triton-server OPENSEARCH_SERVICE := opensearch @@ -96,7 +96,7 @@ ensure-host-bind-mount-dirs: ## F-29/F-71: pre-create bind-mount source dirs as .PHONY: up up: ensure-host-bind-mount-dirs ## Start the core stack (Triton + API + OpenSearch) -- see 'make up-monitoring' for Prometheus/Grafana/Loki/dcgm @echo "Starting core services..." - $(COMPOSE) up -d + $(COMPOSE) up -d --remove-orphans @echo "" @echo "Services starting. Check status with: make status" @echo "API available at: http://localhost:$(API_PORT)" @@ -115,7 +115,7 @@ cluster-cpu: ## Recreate the auto-label worker without GPU passthrough (sklearn .PHONY: up-monitoring up-monitoring: ensure-host-bind-mount-dirs ## Start the core stack PLUS monitoring (F-3: opt-in on a shared host -- alloy mounts docker.sock and tails every container, dcgm-exporter reserves all GPUs) @echo "Starting core + monitoring services..." - $(COMPOSE) --profile monitoring up -d + $(COMPOSE) --profile monitoring up -d --remove-orphans @echo "" @echo "Services starting. Check status with: make status" @echo "API available at: http://localhost:$(API_PORT)" @@ -123,11 +123,11 @@ up-monitoring: ensure-host-bind-mount-dirs ## Start the core stack PLUS monitori .PHONY: dev-up dev-up: ensure-host-bind-mount-dirs ## Single canonical dev deployment: dev overlay + GPU arbiter overlay + curation/segmenter/vlm/training profiles - $(DEV_COMPOSE) up -d --build + $(DEV_COMPOSE) up -d --build --remove-orphans .PHONY: dev-monitoring-up dev-monitoring-up: ensure-host-bind-mount-dirs ## 'make dev-up' PLUS the monitoring profile (Prometheus/Grafana/Loki/Alloy/DCGM + Dashboards); opt-in as in up-monitoring - $(DEV_COMPOSE) --profile monitoring up -d --build + $(DEV_COMPOSE) --profile monitoring up -d --build --remove-orphans .PHONY: dev-down dev-down: ## Stop the dev deployment started by 'make dev-up' (keeps volumes) @@ -135,7 +135,7 @@ dev-down: ## Stop the dev deployment started by 'make dev-up' (keeps volumes) .PHONY: dev-restart dev-restart: ## Restart the code-mounting containers (api + workers) after pulling/merging code; does not recreate or re-read .env - $(DEV_COMPOSE) restart yolo-api curation-detection-worker curation-vlm-worker curation-auto-label-worker curation-cluster-refresh + $(DEV_COMPOSE) restart api curation-detection-worker curation-vlm-worker curation-auto-label-worker curation-cluster-refresh .PHONY: dev-ps dev-ps: ## Show the dev deployment's containers diff --git a/README.md b/README.md index 2169afe3a..10b73c49f 100644 --- a/README.md +++ b/README.md @@ -165,7 +165,7 @@ address also narrows Cropwright to that interface. See ```bash cd openprocessor ./openprocessor status # services and health -./openprocessor logs yolo-api -f # live logs +./openprocessor logs api -f # live logs ./openprocessor sample coco # fetch a public COCO sample (200 images) ./openprocessor vlm list # the local VLM catalog ./openprocessor upgrade # to the latest release (backs up first) @@ -271,14 +271,14 @@ Government Work"; never NonCommercial or NoDerivs). Then: # (PUT /curation/projects/{project}/ingest/policy); the old # OP_INGEST_PRIMARY_CLASS_IDS env var is retired and ignored. # OP_INGEST_PRIMARY_DETECTOR_MODEL picks the detector; to use another -# model set it in .env and recreate yolo-api (see docs/CURATION.md). +# model set it in .env and recreate api (see docs/CURATION.md). # 3. Point OP_SOURCE_ROOT_HOST at data/samples in .env (the compose mount # target is fixed at /data/source): echo 'OP_SOURCE_ROOT_HOST=./data/samples' >> .env -docker compose up -d --force-recreate yolo-api +docker compose up -d --force-recreate api # 4. --root is a container path under the /data/source mount, not a # host-relative one: -docker compose exec yolo-api python scripts/curation/ingest_walker.py \ +docker compose exec api python scripts/curation/ingest_walker.py \ --root /data/source/coco_va_readme/images --project cars \ --api-base http://localhost:8000/curation ``` @@ -291,7 +291,7 @@ to stop (volumes are kept). Set `COMPOSE_PROJECT_NAME`, `OP_IMAGE_REPO` and published ones. In a source checkout use `make up` and `make` targets, which add the dev -overlay; in an installed directory run `./openprocessor restart yolo-api` after +overlay; in an installed directory run `./openprocessor restart api` after editing `.env`. `make sample-coco` (800 images plus side sets), `make sample-plates` (300 Open Images V7 images) and `make sample-clean` use the same tool. See @@ -432,7 +432,7 @@ YOLO26. (`export/export_yolo26.py`, natively NMS-free engines): ```bash - docker compose exec yolo-api python /app/export/export_yolo26.py --models small + docker compose exec api python /app/export/export_yolo26.py --models small curl -X POST http://localhost:4603/models/yolo26_small_trt/load ``` @@ -750,7 +750,7 @@ Client (Port 4603) | v +----------+ +-----------------------------+ - | yolo-api |---->| curation workers (profile) | + | api |---->| curation workers (profile) | +----------+ | detection, VLM, auto-label, | | | cluster refresh, evaluator | v +-----------------------------+ @@ -762,7 +762,7 @@ Client (Port 4603) **Services** (compose profile in brackets): -- `yolo-api` (4603): FastAPI service handling all requests. +- `api` (4603): FastAPI service handling all requests. - `triton-server` (4600-4602): Triton with TensorRT models. - `opensearch` (4607): vector database and the curation datastore. - `curation-detection-worker`, `curation-vlm-worker`, @@ -864,15 +864,15 @@ running stack over HTTP, so they read their target ports from the environment .venv/bin/python tests/validate_visual_results.py 2>&1 | tee test_results/visual_validation.txt ``` -**Docker-only path.** The production `yolo-api` image installs only +**Docker-only path.** The production `api` image installs only `requirements.txt` (no `pytest`, no `requirements-test.txt`), so a bare -`docker compose exec yolo-api pytest ...` fails with `executable file not +`docker compose exec api pytest ...` fails with `executable file not found`. Install the test deps into the running container first (not persisted across a recreate): ```bash -docker compose exec yolo-api pip install -r requirements-test.txt -docker compose exec yolo-api python -m pytest tests/ -q --ignore=tests/live +docker compose exec api pip install -r requirements-test.txt +docker compose exec api python -m pytest tests/ -q --ignore=tests/live ``` See [CONTRIBUTING.md](CONTRIBUTING.md) for the live write-path harness and the diff --git a/benchmarks/README.md b/benchmarks/README.md index 154d9fdc1..56d34ebb2 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -69,7 +69,7 @@ For custom benchmarks with detailed metrics: ```bash # Run inside container -docker compose exec yolo-api python /app/benchmarks/scripts/benchmark.py +docker compose exec api python /app/benchmarks/scripts/benchmark.py # Or use make target make bench-python diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml index 41a11bca3..317d94988 100644 --- a/docker-compose.dev.yml +++ b/docker-compose.dev.yml @@ -33,7 +33,7 @@ services: context: . dockerfile: Dockerfile.triton - yolo-api: + api: build: context: . dockerfile: Dockerfile diff --git a/docker-compose.gpu-arbiter.yml b/docker-compose.gpu-arbiter.yml index 4256fb848..0261cf672 100644 --- a/docker-compose.gpu-arbiter.yml +++ b/docker-compose.gpu-arbiter.yml @@ -1,4 +1,4 @@ -# Opt-in overlay: mounts the host Docker socket into yolo-api so the GPU +# Opt-in overlay: mounts the host Docker socket into api so the GPU # arbiter (src/services/training/gpu_arbiter.py) can stop/restart # GPU-resident sibling containers around a training run # (OP_GPU_ARBITER_CONTAINERS, OP_GPU_ARBITER_TRAINER_CONTAINER -- see @@ -11,10 +11,10 @@ # /var/run/docker.sock into any container gives that container's process # root-equivalent control over the whole Docker host -- it can start, # stop or inspect *any* container, not just the ones this project owns, -# and a container-escape bug in yolo-api's dependency stack becomes a +# and a container-escape bug in api's dependency stack becomes a # host-level compromise. This is why it's a separate, explicitly opt-in # file instead of a default mount. Only enable it on a host you trust -# yolo-api's dependency stack on, and prefer scoping +# api's dependency stack on, and prefer scoping # OP_GPU_ARBITER_CONTAINERS to the smallest container list a deployment # actually needs coordinated. # @@ -27,7 +27,7 @@ # OP_GPU_ARBITER_TRAINER_CONTAINER= services: - yolo-api: + api: volumes: # NOT :ro -- stop/start calls over the socket need write access; # a read-only bind only restricts the mount point, not what the diff --git a/docker-compose.yml b/docker-compose.yml index 62e9d2797..8b524079f 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -131,7 +131,7 @@ services: networks: - triton_net - yolo-api: + api: image: ${OP_API_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor:${OP_IMAGE_TAG:-0.4.1}} container_name: ${COMPOSE_PROJECT_NAME:-openprocessor}-api logging: *default-logging @@ -156,7 +156,7 @@ services: # / OP_VIZ_JOBS_DIR / OP_PROBE_JOBS_DIR / OP_REGION_DRAIN_STATE_DIR # (all default under /jobs — see env.template). This same shared # mount is what makes state.json/heartbeat/cancel.flag visible to - # every yolo-api --workers process, not just whichever one handled + # every api --workers process, not just whichever one handled # a given request. - openprocessor-jobs:/jobs - ./cache/huggingface:/home/appuser/.cache/huggingface # HuggingFace tokenizers (appuser) @@ -297,7 +297,7 @@ services: condition: service_healthy networks: # G-28: Cropwright's default API_UPSTREAM is http://op-api:8000. The - # public compose's service is named yolo-api; this alias lets + # public compose's service is named api; this alias lets # Cropwright's out-of-the-box default resolve without every deployer # having to override API_UPSTREAM. See docs/CURATION.md "Wiring up # Cropwright". @@ -309,7 +309,7 @@ services: # Cropwright: the labeling frontend, opt-in via `--profile cropwright`. It is # also the gateway (OP_GATEWAY_SUBPATHS=true): the one published port for the # app, the API docs and the monitoring UIs, which then stay on loopback. - # Source lives in ./frontend; nginx proxies the API prefix to yolo-api by + # Source lives in ./frontend; nginx proxies the API prefix to api by # its op-api alias, so the browser only ever talks to this origin. # =================================================================== cropwright: @@ -356,7 +356,7 @@ services: timeout: 5s retries: 3 depends_on: - yolo-api: + api: condition: service_started networks: - triton_net @@ -369,7 +369,7 @@ services: # workers. See docs/design/curation_design_rationale.md. # # Every worker below mounts ./src and ./scripts read-only (:ro) — - # unlike yolo-api's writable hot-reload mounts, these are long-running + # unlike api's writable hot-reload mounts, these are long-running # background processes with no request/response cycle to redeploy # against; a `docker compose --profile curation up -d --force-recreate # ` picks up code changes. @@ -388,13 +388,13 @@ services: required: false volumes: - openprocessor-state:/var/lib/openprocessor - # ST-1: shared with yolo-api -- see that service's mount comment. + # ST-1: shared with api -- see that service's mount comment. - openprocessor-crop-cache:/var/cache/openprocessor/crops - # G-06: same source root, same container path, as yolo-api -- the + # G-06: same source root, same container path, as api -- the # worker resolves item source paths written at ingest time against # this mount. See docs/CURATION.md "Mounting your image source". - ${OP_SOURCE_ROOT_HOST:-./data/source}:/data/source:ro - # W9.9: VLM API keys by reference (see yolo-api's mount). + # W9.9: VLM API keys by reference (see api's mount). - ./secrets/vlm:/run/secrets/op_vlm:ro # G-08: region profile examples (OP_REGION_PROFILE_PATH) ship baked # into the image (see Dockerfile); docker-compose.dev.yml overrides @@ -420,7 +420,7 @@ services: # S-3: lets the worker publish crop.region_verified events to every # subscribed SSE client via POST {api_prefix}/events/publish, not # just whichever uvicorn worker happened to handle a given POST. - - OP_EVENT_API_URL=http://yolo-api:8000 + - OP_EVENT_API_URL=http://api:8000 command: - python - -m @@ -455,9 +455,9 @@ services: volumes: - openprocessor-state:/var/lib/openprocessor environment: - - OP_API=http://yolo-api:8000 + - OP_API=http://api:8000 - OPENSEARCH_URL=http://opensearch:9200 - # G-10: matches yolo-api's extra_hosts -- lets OP_VLM_URL point at + # G-10: matches api's extra_hosts -- lets OP_VLM_URL point at # http://host.docker.internal:PORT/v1 for an external VLM on Linux. extra_hosts: - "host.docker.internal:host-gateway" @@ -473,7 +473,7 @@ services: start_period: 60s retries: 3 depends_on: - yolo-api: + api: condition: service_healthy opensearch: condition: service_started @@ -492,7 +492,7 @@ services: - path: ./.env required: false volumes: - # Shared with yolo-api: this is the trigger/state/heartbeat file + # Shared with api: this is the trigger/state/heartbeat file # protocol POST /curation/pipeline/auto_label writes into (see # OP_AUTO_LABEL_STATE_DIR, default /jobs/auto_label). - openprocessor-jobs:/jobs @@ -500,7 +500,7 @@ services: # in-process (pipeline -> _common -> cascade_detect -> # profile_registry / class registry / source-image serving), not a # thin HTTP client like vlm-worker/cluster-refresh -- it needs the - # same three mounts as curation-detection-worker and yolo-api, at + # same three mounts as curation-detection-worker and api, at # the same container paths, or it 404s/FileNotFoundErrors on its # first claimed job while its idle healthcheck stays green. # Source images (OP_SOURCE_ROOT): without this every thumbnail @@ -511,7 +511,7 @@ services: # without this the registry loads empty and the stage raises # "class_names or class_catalog must be supplied". - ./data:/app/data - # W9.9: VLM API keys by reference (see yolo-api's mount): this worker + # W9.9: VLM API keys by reference (see api's mount): this worker # builds the VLM labeler in-process for the auto-label VLM stage. - ./secrets/vlm:/run/secrets/op_vlm:ro # Region profiles (OP_REGION_PROFILE_PATH): without this the worker @@ -558,7 +558,7 @@ services: command: - python - scripts/curation/cluster_refresh_daemon.py - - --api=http://yolo-api:8000 + - --api=http://api:8000 - --opensearch=http://opensearch:9200 healthcheck: test: @@ -571,7 +571,7 @@ services: start_period: 60s retries: 3 depends_on: - yolo-api: + api: condition: service_healthy opensearch: condition: service_started @@ -598,7 +598,7 @@ services: - MLFLOW_TRACKING_URI=http://curation-mlflow:5000 - MLFLOW_EXPERIMENT_NAME=${MLFLOW_EXPERIMENT_NAME:-openprocessor} volumes: - # Exports are eval datasets: same /app/data path as yolo-api, where + # Exports are eval datasets: same /app/data path as api, where # the API resolved them (per project, under ./data/projects//exports). - ./data:/app/data:ro - openprocessor-state:/var/lib/openprocessor @@ -635,7 +635,7 @@ services: ports: - "${OP_BIND_ADDRESS:-127.0.0.1}:${SEGMENTER_PORT:-4611}:8000" # Segmentation HTTP API volumes: - - ./cache/huggingface:/home/appuser/.cache/huggingface # ST-2: nvme-backed HF cache (appuser, matches yolo-api) + - ./cache/huggingface:/home/appuser/.cache/huggingface # ST-2: nvme-backed HF cache (appuser, matches api) environment: # SEGMENTER_LISTEN_PORT is the container's own uvicorn bind port and # is intentionally NOT the same name as SEGMENTER_PORT (the host-side @@ -702,14 +702,14 @@ services: - path: ./.env required: false volumes: - # Shared job-file protocol with yolo-api (OP_TRAIN_JOBS_DIR). + # Shared job-file protocol with api (OP_TRAIN_JOBS_DIR). - openprocessor-jobs:/jobs # Run outputs (OP_TRAIN_RUNS_ROOT) + bake-off job drop. This MUST be - # mounted at the same absolute path here and in yolo-api: the trainer + # mounted at the same absolute path here and in api: the trainer # records `checkpoint_path` verbatim and promote resolves that exact # string from the API container. - openprocessor-state:/var/lib/openprocessor - # Frozen dataset exports. Mounted at the same path yolo-api uses so the + # Frozen dataset exports. Mounted at the same path api uses so the # absolute `dataset_export_dir` in job.json resolves identically. - ./data:/app/data:ro # Triton model repo, read-only — only so the incumbent comparison can @@ -722,7 +722,7 @@ services: # names HOST gpu ids, which appear inside the container as positional # local indices. See docker/trainer/trainer.py::resolve_local_device. - OP_TRAIN_GPU_ORDER=${OP_TRAIN_GPU_ORDER:-0} - - OP_API_BASE_URL=http://yolo-api:8000 + - OP_API_BASE_URL=http://api:8000 - OP_API_PREFIX=${OP_API_PREFIX:-/curation} - OP_TRITON_HTTP_URL=http://triton-server:8000 - OP_TRITON_MODEL_REPO=/models @@ -745,7 +745,7 @@ services: capabilities: - gpu depends_on: - - yolo-api + - api networks: - triton_net @@ -803,7 +803,7 @@ services: # G-10: optional in-compose VLM (Gemma 4 E4B) for the labeling/verify # legs (OP_VLM_URL). Off by default -- most deployments either have no # VLM yet or already run one elsewhere (set OP_VLM_URL to point at it; - # see the host.docker.internal extra_hosts on yolo-api / + # see the host.docker.internal extra_hosts on api / # curation-vlm-worker for "VLM runs on the host, not in this compose # project"). Enable with `docker compose --profile vlm up -d vlm` and # set OP_VLM_URL=http://vlm:8000/v1 in .env. @@ -1133,7 +1133,7 @@ volumes: openprocessor-state: openprocessor-jobs: mlflow_data: - # ST-1: shared tmpfs item-crop cache (yolo-api + curation-detection-worker). + # ST-1: shared tmpfs item-crop cache (api + curation-detection-worker). # Size is the OS-level ceiling; OP_CROP_CACHE_MAX_BYTES (default 3 GiB, # see CurationConfig.crop_cache_max_bytes) prunes the *content* down # before this fills. diff --git a/docker/trainer/Dockerfile b/docker/trainer/Dockerfile index edc8b76e9..4ad8c1103 100644 --- a/docker/trainer/Dockerfile +++ b/docker/trainer/Dockerfile @@ -21,7 +21,7 @@ # the export root -> frozen dataset exports (read-only) # # IMPORTANT: the runs dir must be mounted at the SAME absolute path in this -# container and in yolo-api. The trainer records `checkpoint_path` verbatim in +# container and in api. The trainer records `checkpoint_path` verbatim in # status.json, and promote resolves that exact string from the API container. # # Build: diff --git a/docker/trainer/campaign.py b/docker/trainer/campaign.py index 9e063363b..8a79d3bd9 100644 --- a/docker/trainer/campaign.py +++ b/docker/trainer/campaign.py @@ -44,7 +44,7 @@ # API base + prefix for the campaign auto-promote and auto-quantize callbacks. The trainer and the # API share a compose network, so the service name resolves via Docker DNS. -API_BASE_URL = os.environ.get('OP_API_BASE_URL', 'http://yolo-api:8000') +API_BASE_URL = os.environ.get('OP_API_BASE_URL', 'http://api:8000') API_PREFIX = os.environ.get('OP_API_PREFIX', '/curation') diff --git a/docs-site/architecture-diagrams/README.md b/docs-site/architecture-diagrams/README.md index 19818c838..31faca639 100644 --- a/docs-site/architecture-diagrams/README.md +++ b/docs-site/architecture-diagrams/README.md @@ -34,7 +34,7 @@ validation is never silently rendered broken. | Group | id | Type | What it shows | | --------- | ------------------- | ------------ | ------------- | -| System | `system-overview` | architecture | Every compose service and profile: clients, yolo-api, Triton, OpenSearch, the job volumes, the multi-project curation workers, segmenter, VLM endpoints, the host CLI, trainer, evaluator, MLflow and monitoring. | +| System | `system-overview` | architecture | Every compose service and profile: clients, api, Triton, OpenSearch, the job volumes, the multi-project curation workers, segmenter, VLM endpoints, the host CLI, trainer, evaluator, MLflow and monitoring. | | System | `backend-modules` | architecture | `src/main.py`, core and project-scoped curation routers, the core/curation/detection/labeling/training/projects services, the config store, clients, `src/config/`, and the workers that import the services. | | System | `data-model` | architecture | A project's six indexes (items carry `region_boxes`) and its class registry and exports, the global `op_projects` and `op_global_configs` indexes, and the separate visual-search indexes. | | Workflows | `core-inference` | workflow | `/detect`, `/faces`, `/embed` and `/ocr`: CPU preprocessing in the API, TensorRT models in Triton, and the OCR BLS pipeline. | diff --git a/docs-site/architecture-diagrams/specs/curation-pipeline.workflow.json b/docs-site/architecture-diagrams/specs/curation-pipeline.workflow.json index c8aec80ee..7a8ddf877 100644 --- a/docs-site/architecture-diagrams/specs/curation-pipeline.workflow.json +++ b/docs-site/architecture-diagrams/specs/curation-pipeline.workflow.json @@ -67,7 +67,7 @@ "lanes": [ { "id": "ingest", - "label": "Ingest (yolo-api)" + "label": "Ingest (api)" }, { "id": "workers", diff --git a/docs-site/architecture-diagrams/specs/system-overview.architecture.json b/docs-site/architecture-diagrams/specs/system-overview.architecture.json index 5765ade6c..332941f4e 100644 --- a/docs-site/architecture-diagrams/specs/system-overview.architecture.json +++ b/docs-site/architecture-diagrams/specs/system-overview.architecture.json @@ -99,7 +99,7 @@ { "id": "api", "type": "backend", - "label": "yolo-api", + "label": "api", "sublabel": "FastAPI :4603", "pos": [ 280, @@ -359,7 +359,7 @@ "dot": "cyan", "title": "Core (always on)", "items": [ - "yolo-api, triton-server and opensearch are the only services without a compose profile", + "api, triton-server and opensearch are the only services without a compose profile", "One shared Triton gRPC client lets Triton's dynamic batcher form batches", "No authentication on any route: run behind your own proxy" ] diff --git a/docs-site/architecture-diagrams/specs/training-promote.sequence.json b/docs-site/architecture-diagrams/specs/training-promote.sequence.json index 30a94d05f..1236fc343 100644 --- a/docs-site/architecture-diagrams/specs/training-promote.sequence.json +++ b/docs-site/architecture-diagrams/specs/training-promote.sequence.json @@ -55,7 +55,7 @@ { "id": "api", "type": "backend", - "label": "yolo-api", + "label": "api", "sublabel": "{project}/train/*" }, { diff --git a/docs-site/docs/architecture/overview.mdx b/docs-site/docs/architecture/overview.mdx index cb2a50197..c66590407 100644 --- a/docs-site/docs/architecture/overview.mdx +++ b/docs-site/docs/architecture/overview.mdx @@ -11,7 +11,7 @@ highlight one part of it. | Group | Diagram | What it shows | | --- | --- | --- | -| System | System overview | Every compose service and profile: clients, `yolo-api`, Triton, OpenSearch, the shared job volumes, the multi-project curation workers, segmenter, VLM endpoints, trainer, evaluator, MLflow and monitoring. | +| System | System overview | Every compose service and profile: clients, `api`, Triton, OpenSearch, the shared job volumes, the multi-project curation workers, segmenter, VLM endpoints, trainer, evaluator, MLflow and monitoring. | | System | Backend modules | `src/main.py`, the core and curation routers (project-scoped), the FastAPI-free services under `src/services/` (projects, config store, dataset import, reprocess, combine), the clients, `src/config/`, and the workers that import the same services. | | System | Data model | The per-project curation indexes (images, items with their `region_boxes`, classes, UMAP state and the `configs` store), the global project and config indexes, the project's files, and the separate visual-search indexes. | | Workflows | Core inference | `/detect`, `/faces`, `/embed` and `/ocr`: CPU preprocessing in the API, TensorRT models in Triton, and the `ocr_pipeline` BLS model. | diff --git a/docs-site/docs/configuration/advanced.mdx b/docs-site/docs/configuration/advanced.mdx index 96113ca9f..0d2440723 100644 --- a/docs-site/docs/configuration/advanced.mdx +++ b/docs-site/docs/configuration/advanced.mdx @@ -83,7 +83,7 @@ checkout rarely needs them. | Variable | Default | Meaning | |---|---|---| -| `OP_SOURCE_ROOT` | `/data/source` | Container path item paths resolve against; must match the mount on `yolo-api` and the detection worker | +| `OP_SOURCE_ROOT` | `/data/source` | Container path item paths resolve against; must match the mount on `api` and the detection worker | | `OP_SOURCE_ROOT_HOST` | `./data/source` | Host side of that mount | | `OP_SOURCE_PATH_ALIASES` | unset | Extra named roots, JSON object or `alias=path,...`, served at `GET /curation/projects/{project}/images/root/{alias}` | | `OP_EXPORT_KEEP_LAST` | `5` | Keep-last retention for auto-named exports under each project's exports directory; `0` keeps all | @@ -242,7 +242,7 @@ See [Dataset import](../guides/dataset-import.mdx) and | `OP_TRAIN_COMPARE_MAX_IMAGES` / `OP_TRAIN_COMPARE_INPUT_SIZE` | `200` / `640` | | | `OP_TRITON_MODEL_REPO` | `/models` | Must match the directory Triton serves, or a promoted model lands where Triton never looks | | `OP_TRITON_HTTP_URL` | `http://triton-server:8000` | | -| `OP_API_BASE_URL` | `http://yolo-api:8000` | Where the trainer posts a campaign's auto-promote | +| `OP_API_BASE_URL` | `http://api:8000` | Where the trainer posts a campaign's auto-promote | | `OP_TRAIN_LOG_LEVEL` | `INFO` | | | `MLFLOW_TRACKING_URI`, `MLFLOW_EXPERIMENT_NAME`, `OP_MLFLOW_MODEL_NAME`, `OP_MLFLOW_CONFUSION_MATRIX_EVERY` | | MLflow; every hook degrades to a warning when unreachable. A project's runs use the experiment `openprocessor-{slug}` | | `OP_MLFLOW_PUBLIC_URL` | unset | Browser-reachable MLflow base for served `mlflow_run_url` | diff --git a/docs-site/docs/configuration/basic.mdx b/docs-site/docs/configuration/basic.mdx index 83c2cba66..2a6d407a7 100644 --- a/docs-site/docs/configuration/basic.mdx +++ b/docs-site/docs/configuration/basic.mdx @@ -29,7 +29,7 @@ another. :::warning[`OP_*` variables are read at process start] Curation routers build their prefix when `src.main` is imported, and workers read their settings once at startup. Changing any `OP_*` variable -needs a container recreate (`docker compose up -d --force-recreate yolo-api`), +needs a container recreate (`docker compose up -d --force-recreate api`), not just a reload. ::: @@ -70,7 +70,7 @@ values by hand. | `OPENSEARCH_HEAP` | `2g` | OpenSearch JVM heap. The installer sets it from host RAM (RAM/8, 1-8 GB) and never overwrites a value you set. Raise it for more projects: about 1 GB per 20 shards | If a generated `docker-compose.override.yml` exists, its GPU pinning for -`triton-server` and `yolo-api` wins over the `*_GPU_ID` variables below. +`triton-server` and `api` wins over the `*_GPU_ID` variables below. ## GPU ids @@ -80,7 +80,7 @@ variable; see [GPU sizing](../deployment/gpu-sizing.mdx#gpu-ids). | Variable | Default | Service | |---|---|---| | `TRITON_GPU_ID` | `0` | `triton-server` | -| `API_GPU_ID` | `0` | `yolo-api` (torch fallbacks, CUDA-side preprocessing) | +| `API_GPU_ID` | `0` | `api` (torch fallbacks, CUDA-side preprocessing) | | `EVALUATOR_GPU_ID` | `0` | `curation-evaluator` | | `SEGMENTER_GPU_ID` | `0` | `segmenter` | | `VLM_GPU_ID` | `0` | `vlm` | @@ -139,7 +139,7 @@ curation deployment actually sets: | `OP_INGEST_PRIMARY_DETECTOR_MODEL` | **Required for ingest** (it `503`s without one): the Triton end2end model that proposes items, e.g. `yolov11_small_trt_end2end` | | `OP_INGEST_PRIMARY_CLASS_IDS` | Retired and ignored with a warning. Narrow ingest per project with the ingest policy instead | | `OP_INGEST_PRIMARY_ASSIGNS_CLASS` | `false` (default): detections are unlabeled proposals. `true` only if the detector was trained on your class registry | -| `OP_SOURCE_ROOT_HOST` | Host directory mounted read-only at `/data/source` in `yolo-api` and the detection worker (default `./data/source`) | +| `OP_SOURCE_ROOT_HOST` | Host directory mounted read-only at `/data/source` in `api` and the detection worker (default `./data/source`) | | `OP_VLM_URL`, `OP_VLM_MODEL`, `OP_VLM_API_KEY` | An OpenAI-compatible `/v1/chat/completions` endpoint. This defines the built-in `env` VLM endpoint; `OP_VLM_MODEL` is required whenever `OP_VLM_URL` is set. More endpoints are registered through the API; see [VLM selection](../guides/vlm-selection.mdx) | | `OP_SEGMENTER_URL` | e.g. `http://segmenter:8000`; empty makes the cascade's segmenter leg a clean no-op | | `HF_TOKEN` | Needed by the segmenter (gated SAM 3 weights) and for gated VLM weights | diff --git a/docs-site/docs/deployment/gpu-sizing.mdx b/docs-site/docs/deployment/gpu-sizing.mdx index 56697a34b..3f3ca3bed 100644 --- a/docs-site/docs/deployment/gpu-sizing.mdx +++ b/docs-site/docs/deployment/gpu-sizing.mdx @@ -101,7 +101,7 @@ spread the load: | Variable | Service | |---|---| | `TRITON_GPU_ID` | `triton-server` | -| `API_GPU_ID` | `yolo-api` | +| `API_GPU_ID` | `api` | | `EVALUATOR_GPU_ID` | `curation-evaluator` | | `SEGMENTER_GPU_ID` | `segmenter` | | `VLM_GPU_ID` | `vlm` | diff --git a/docs-site/docs/deployment/security.mdx b/docs-site/docs/deployment/security.mdx index ad5a4b0eb..91bc494b5 100644 --- a/docs-site/docs/deployment/security.mdx +++ b/docs-site/docs/deployment/security.mdx @@ -91,7 +91,7 @@ cover. ## The Docker socket overlay `docker-compose.gpu-arbiter.yml` mounts `/var/run/docker.sock` into -`yolo-api` so training jobs can stop and restart named containers around a +`api` so training jobs can stop and restart named containers around a run (`OP_GPU_ARBITER_CONTAINERS`). Mounting the socket gives that container's processes **root-equivalent control of the whole Docker host**, not just this project. Only enable it on a host where you trust diff --git a/docs-site/docs/getting-started/compose-profiles.mdx b/docs-site/docs/getting-started/compose-profiles.mdx index 0fe52087b..d7f15430c 100644 --- a/docs-site/docs/getting-started/compose-profiles.mdx +++ b/docs-site/docs/getting-started/compose-profiles.mdx @@ -11,7 +11,7 @@ exactly Triton, the API and OpenSearch. Everything else is opt-in. | Profile | Services | Needs a GPU | Start with | |---|---|---|---| -| *(none)* | `triton-server`, `yolo-api`, `opensearch` | Triton, and the API for torch fallbacks | `docker compose up -d` / `make up` | +| *(none)* | `triton-server`, `api`, `opensearch` | Triton, and the API for torch fallbacks | `docker compose up -d` / `make up` | | `curation` | `curation-detection-worker`, `curation-vlm-worker`, `curation-auto-label-worker`, `curation-cluster-refresh`, `curation-evaluator` | only the evaluator | `docker compose --profile curation up -d` / `make curation-up` | | `segmenter` | `segmenter` (SAM 3 behind the generic segment protocol) | yes, plus `HF_TOKEN` for the gated weights | `docker compose --profile curation --profile segmenter up -d` | | `vlm` | `vlm` (vLLM, OpenAI-compatible) | yes | `docker compose --profile vlm up -d` | @@ -34,7 +34,7 @@ Notes: - **The `vlm` profile serves one model at a time**, chosen from a catalog with `openprocessor vlm use `; see [VLM selection](../guides/vlm-selection.mdx). `./secrets/vlm` is mounted - read-only into `yolo-api` and the workers so endpoints can reference keys + read-only into `api` and the workers so endpoints can reference keys as `secret:`. - **`curation-evaluator` is a long-lived watcher**, not a one-shot job. Start it with `up -d`; `docker compose run --rm curation-evaluator` @@ -47,15 +47,15 @@ Notes: - **Docker socket for the GPU arbiter** is a separate overlay file, `docker-compose.gpu-arbiter.yml`, not a profile. It lets training jobs stop and restart named sibling containers around a run, at the cost of - giving `yolo-api` root-equivalent control of the Docker host. See + giving `api` root-equivalent control of the Docker host. See [Security](../deployment/security.mdx). ## Wiring a frontend to the stack The compose project creates one network, -`${COMPOSE_PROJECT_NAME:-openprocessor}_triton_net`. `yolo-api` carries +`${COMPOSE_PROJECT_NAME:-openprocessor}_triton_net`. `api` carries the network alias `op-api`, so a frontend on that network reaches the API -at `http://op-api:8000` (or `http://yolo-api:8000`). For Cropwright: +at `http://op-api:8000` (or `http://api:8000`). For Cropwright: | Cropwright variable | Value | |---|---| diff --git a/docs-site/docs/getting-started/introduction.mdx b/docs-site/docs/getting-started/introduction.mdx index b116871aa..b8a5cc34d 100644 --- a/docs-site/docs/getting-started/introduction.mdx +++ b/docs-site/docs/getting-started/introduction.mdx @@ -6,7 +6,7 @@ title: Introduction # Introduction OpenProcessor is a **self-hosted visual AI backend built on NVIDIA Triton -Inference Server**. One FastAPI service (`yolo-api`, port `4603`) puts +Inference Server**. One FastAPI service (`api`, port `4603`) puts TensorRT-accelerated detection, face recognition, image/text embeddings, OCR and vector search behind a single REST API. An optional, experimental **curation subsystem** adds an active-learning dataset backend on top of @@ -36,7 +36,7 @@ The `/curation` surface has a single, unversioned prefix. See the - **`triton-server`** serves every model from `models/` in explicit model-control mode, so models load, unload and get promoted at runtime. -- **`yolo-api`** does request validation, preprocessing, orchestration and +- **`api`** does request validation, preprocessing, orchestration and response shaping, and talks to Triton over one shared gRPC client so concurrent requests reach Triton's dynamic batcher together. - **`opensearch`** stores vectors and metadata for search, and each diff --git a/docs-site/docs/getting-started/model-export.mdx b/docs-site/docs/getting-started/model-export.mdx index e086d0042..de9f5bc60 100644 --- a/docs-site/docs/getting-started/model-export.mdx +++ b/docs-site/docs/getting-started/model-export.mdx @@ -12,8 +12,8 @@ Triton loads the cached `.plan` files on every later start. it by hand with the Makefile targets, and the models `setup.sh` does not build (PE-Core for curation). -Every target below runs inside the running `yolo-api` container -(`docker compose exec yolo-api ...`), so start the core stack first +Every target below runs inside the running `api` container +(`docker compose exec api ...`), so start the core stack first (`make up`). Weights you download land in `pytorch_models/`, which the API container mounts. @@ -71,7 +71,7 @@ both families; the API reads each model's output format from Triton metadata. ```bash -docker compose exec yolo-api python /app/export/export_yolo26.py --models small +docker compose exec api python /app/export/export_yolo26.py --models small curl -X POST http://localhost:4603/models/yolo26_small_trt/load curl -X POST "http://localhost:4603/detect?model_name=yolo26_small_trt" -F "image=@photo.jpg" ``` @@ -103,7 +103,7 @@ make export-pe | `make pe-build-trt` | Build the TensorRT plan in `models/pe_image_encoder` | | `make pe-export-text-triton` | Export the text tower (parity-gated against PyTorch) and install it as Triton's `pe_text_encoder` | -It then restarts Triton; restart `yolo-api` afterwards. If the TensorRT +It then restarts Triton; restart `api` afterwards. If the TensorRT build fails on PE's attention-pool ops, run `make pe-build-ort` to serve the image ONNX through Triton's ONNX Runtime backend instead. `make pe-text-status` (`GET /health/pe_text`) shows which text backend diff --git a/docs-site/docs/getting-started/quick-start.mdx b/docs-site/docs/getting-started/quick-start.mdx index 56d185ded..98ca19b47 100644 --- a/docs-site/docs/getting-started/quick-start.mdx +++ b/docs-site/docs/getting-started/quick-start.mdx @@ -55,7 +55,7 @@ cd openprocessor curl http://127.0.0.1:4603/health # readiness: Triton + OpenSearch reachable curl -X POST http://127.0.0.1:4603/detect -F "image=@your-image.jpg" ./openprocessor status # service health -./openprocessor logs yolo-api -f # live logs +./openprocessor logs api -f # live logs ./openprocessor sample coco # public COCO sample (200 images) ``` @@ -142,7 +142,7 @@ cat >> .env <<'EOF' OP_INGEST_PRIMARY_DETECTOR_MODEL=yolov11_small_trt_end2end OP_SOURCE_ROOT_HOST=./data/samples EOF -docker compose up -d --force-recreate yolo-api +docker compose up -d --force-recreate api # 3. Create a project, then its classes (the registry starts empty) API=http://localhost:4603 @@ -158,7 +158,7 @@ curl -s -X POST "$API/curation/projects/$P/classes/seed_from_detector" \ # 4. Ingest into the project. --root is a container path under the # /data/source mount -docker compose exec yolo-api python scripts/curation/ingest_walker.py \ +docker compose exec api python scripts/curation/ingest_walker.py \ --project $P --root /data/source/coco_va_readme/images \ --api-base http://localhost:8000/curation @@ -184,14 +184,14 @@ Things that trip people up: `docs/PERFORMANCE.md`). - **Switching detectors.** Set `OP_INGEST_PRIMARY_DETECTOR_MODEL` (and `OP_INGEST_PRIMARY_LABELS_PATH` if the model directory has no `labels.txt`) - in `.env`, then recreate `yolo-api`. The model must be loaded in Triton and + in `.env`, then recreate `api`. The model must be loaded in Triton and serve the end2end four-tensor output. The detector is deployment-wide. - **Ingest returns `503`** until `OP_INGEST_PRIMARY_DETECTOR_MODEL` is set and that model is loaded in Triton. - **`OP_SOURCE_ROOT_HOST`** is the host side of a fixed `/data/source` mount. The walker's `--root` must be a *container* path under it, otherwise every image fails as `unservable_path`. -- **`OP_*` variables are read at process start**: recreate `yolo-api` +- **`OP_*` variables are read at process start**: recreate `api` after changing them. - **`auto_label` clusters only by default** (`run_vlm=false`); the VLM labels nothing unless you pass `run_vlm=true` and a VLM is configured @@ -220,7 +220,7 @@ import can be resumed or undone: ```bash make sample-coco-import # public COCO fixture, four layouts under data/samples/import_fixture -docker compose exec yolo-api python scripts/curation/import_labeled_dataset.py \ +docker compose exec api python scripts/curation/import_labeled_dataset.py \ --project $P --dataset /data/source/import_fixture/yolo \ --accept-suggestions --dry-run ``` diff --git a/docs-site/docs/guides/wheel-example.mdx b/docs-site/docs/guides/wheel-example.mdx index 4f4711c69..473cac750 100644 --- a/docs-site/docs/guides/wheel-example.mdx +++ b/docs-site/docs/guides/wheel-example.mdx @@ -38,7 +38,7 @@ matches an item's `class_name`, or the detector's own label accept the segmenter's boxes unverified. See [VLM selection](./vlm-selection.mdx). Without an active endpoint the boxes stay unverified. -- The image folder is mounted read-only at the same path in `yolo-api` and +- The image folder is mounted read-only at the same path in `api` and in the detection worker. By default `OP_SOURCE_ROOT_HOST` (default `./data/source`) is mounted at `/data/source`. If only the API has the mount, ingest succeeds but every later worker read fails. diff --git a/docs-site/docs/operations/health-and-stalls.mdx b/docs-site/docs/operations/health-and-stalls.mdx index 181a5e514..37a8311d5 100644 --- a/docs-site/docs/operations/health-and-stalls.mdx +++ b/docs-site/docs/operations/health-and-stalls.mdx @@ -101,7 +101,7 @@ An item's region status is derived from its boxes (see - Any path can end in `detection_failed` (for example `reason=image_unavailable` when the worker can't read the source image; check that the detection worker mounts the same source directory as - `yolo-api`). + `api`). - A human can set a box to `false_positive`, which keeps it as a hard negative for training. diff --git a/docs-site/docs/operations/monitoring.mdx b/docs-site/docs/operations/monitoring.mdx index b54b7c3aa..b2f8cc66c 100644 --- a/docs-site/docs/operations/monitoring.mdx +++ b/docs-site/docs/operations/monitoring.mdx @@ -40,7 +40,7 @@ and OpenSearch Dashboards run without authentication, like the API. | Job | Target | Metrics | | --- | --- | --- | | `triton` | `triton-server:8002` | Triton's `nv_*` series: per-model request, queue and compute time, batch execution, pending requests, GPU utilization and memory. | -| `yolo-api` | `yolo-api:8000` | The API's own `/metrics`. | +| `api` | `api:8000` | The API's own `/metrics`. | | `node` | `node-exporter:9100` | Host CPU, memory, load. | | `dcgm` | `dcgm-exporter:9400` | Per-GPU `DCGM_FI_DEV_*` series. | | `loki` | `loki:3100` | Loki's own health. | @@ -56,7 +56,7 @@ change a port, change the matching job in `prometheus.yml`). @@ -135,7 +135,7 @@ filter on those labels. Loki keeps its data in the `loki_data` volume. :::caution[Check the container filter] Alloy's keep rules match container names that start with `triton-server` -or `yolo-api`, while the compose file names containers +or `api`, while the compose file names containers `-triton` and `-api`. Because Alloy reads the host's Docker socket, it also sees containers from other compose projects on the same host. Run `curl -s localhost:4606/loki/api/v1/label/container/values` diff --git a/docs-site/docs/operations/workers.mdx b/docs-site/docs/operations/workers.mdx index 498927327..7365be5c6 100644 --- a/docs-site/docs/operations/workers.mdx +++ b/docs-site/docs/operations/workers.mdx @@ -36,7 +36,7 @@ make curation-down | `curation-cluster-refresh` | `scripts/curation/cluster_refresh_daemon.py` | Polls each project's items index (every 2 minutes by default) and calls that project's `POST /curation/projects/{project}/clusters/auto_promote` once it has grown by the threshold (1000 new items by default); optional incremental auto-label. | | `curation-evaluator` | its own image | Watches each project's `bakeoff_jobs` directory and runs model comparisons. Long-lived: start with `up -d`, not `run --rm`. | -The detection worker and `yolo-api` must mount the **same** source +The detection worker and `api` must mount the **same** source directory at the same container path (`OP_SOURCE_ROOT_HOST` → `/data/source`). If only the API has it, ingest succeeds but every worker read fails with `detection_failed` / `image_unavailable`. @@ -89,7 +89,7 @@ There are two independent pauses. ## Multi-process state -`yolo-api` runs many uvicorn worker processes, and any of them can answer +`api` runs many uvicorn worker processes, and any of them can answer a given request. Every piece of job state that must look the same from all of them lives in files on the shared `/jobs` volume, not in process memory: diff --git a/docs-site/src/data/architecture-diagrams.json b/docs-site/src/data/architecture-diagrams.json index d00f30541..8099326a1 100644 --- a/docs-site/src/data/architecture-diagrams.json +++ b/docs-site/src/data/architecture-diagrams.json @@ -6,7 +6,7 @@ { "id": "system-overview", "title": "System Overview", - "description": "Every service in docker-compose.yml and how they talk: clients reach yolo-api, which calls Triton over gRPC and OpenSearch; the opt-in profiles add the multi-project curation workers, segmenter, VLM endpoints, trainer, evaluator, MLflow and monitoring. Start here.", + "description": "Every service in docker-compose.yml and how they talk: clients reach api, which calls Triton over gRPC and OpenSearch; the opt-in profiles add the multi-project curation workers, segmenter, VLM endpoints, trainer, evaluator, MLflow and monitoring. Start here.", "height": 1000 }, { diff --git a/docs-site/src/data/backend_screenshots.json b/docs-site/src/data/backend_screenshots.json index 4345f32bf..883cf2e30 100644 --- a/docs-site/src/data/backend_screenshots.json +++ b/docs-site/src/data/backend_screenshots.json @@ -21,7 +21,7 @@ }, { "name": "backend-prometheus-targets.png", - "alt": "Prometheus target health page with the triton, yolo-api, node, dcgm and loki scrape targets all up", + "alt": "Prometheus target health page with the triton, api, node, dcgm and loki scrape targets all up", "caption": "Prometheus: Triton, API, node, GPU (DCGM) and Loki targets, all up" }, { diff --git a/docs-site/static/architecture/curation-pipeline.html b/docs-site/static/architecture/curation-pipeline.html index 5d96861fd..a9a832e96 100644 --- a/docs-site/static/architecture/curation-pipeline.html +++ b/docs-site/static/architecture/curation-pipeline.html @@ -5031,7 +5031,7 @@

Curation Pipeline

- 01 / Ingest (yolo-api) + 01 / Ingest (api) 02 / Curation workers @@ -5066,8 +5066,8 @@

Curation Pipeline

- - Project · /curation/projects/{project} · Ingest (yolo-api) + + Project · /curation/projects/{project} · Ingest (api) - - Ingest / import · /ingest/*, /datasets/imports · Ingest (yolo-api) + + Ingest / import · /ingest/*, /datasets/imports · Ingest (api) - - Item detector · OP_INGEST_PRIMARY_* · Ingest (yolo-api) + + Item detector · OP_INGEST_PRIMARY_* · Ingest (api) - - PE-Core embed · pe_embedding 1024-d · Ingest (yolo-api) + + PE-Core embed · pe_embedding 1024-d · Ingest (api) - - yolo-api · FastAPI :4603 · Architecture component · core + /curation + + api · FastAPI :4603 · Architecture component · core + /curation - yolo-api + api FastAPI :4603 core + /curation @@ -5476,7 +5476,7 @@

OpenProcessor System Overview

Core (always on)

    -
  • • yolo-api, triton-server and opensearch are the only services without a compose profile
    • +
  • • api, triton-server and opensearch are the only services without a compose profile
  • • One shared Triton gRPC client lets Triton's dynamic batcher form batches
  • • No authentication on any route: run behind your own proxy
diff --git a/docs-site/static/architecture/training-promote.html b/docs-site/static/architecture/training-promote.html index 1105711ba..15dee6592 100644 --- a/docs-site/static/architecture/training-promote.html +++ b/docs-site/static/architecture/training-promote.html @@ -5220,14 +5220,14 @@

Training and Promote

curl / labeling UI - - yolo-api · {project}/train/* · Sequence participant + + api · {project}/train/* · Sequence participant - yolo-api + api {project}/train/* diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 6b09c0cb9..b5631dd79 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -35,7 +35,7 @@ wire contract see [design/curation_api_contract.md](design/curation_api_contract | v :4603 +-----------+ gRPC +----------------+ - | yolo-api |-------->| triton-server | TensorRT engines (GPU) + | api |-------->| triton-server | TensorRT engines (GPU) | (FastAPI) | +----------------+ | | HTTP +----------------+ | |-------->| opensearch | k-NN + curation datastore @@ -76,7 +76,7 @@ checkout. GPU placement comes from `.env` keys, not from the compose files. | Service | Profile | Port | Role | |---|---|---|---| | `triton-server` | none | 4600 HTTP, 4601 gRPC, 4602 metrics | Serves the TensorRT models. Explicit model control; loads the list in the compose command; a missing engine leaves that model unloaded instead of killing the server | -| `yolo-api` | none | 4603 | FastAPI, uvicorn with 32 worker processes | +| `api` | none | 4603 | FastAPI, uvicorn with 32 worker processes | | `opensearch` | none | 4607 | k-NN indexes and every curation document | | `curation-detection-worker` | `curation` | | Region cascade over pending items, every active project in turn | | `curation-vlm-worker` | `curation` | | VLM class and verification loop | @@ -96,7 +96,7 @@ Per-project class registries, exports and bake-off data live under `./data/projects//`. `docker-compose.gpu-arbiter.yml` is an opt-in overlay that mounts the Docker -socket into `yolo-api` so the GPU arbiter can stop and restart sibling +socket into `api` so the GPU arbiter can stop and restart sibling containers around a training run. It gives that container control of the Docker host; read its header before using it. diff --git a/docs/CURATION.md b/docs/CURATION.md index 5ce30e6c5..b23377519 100644 --- a/docs/CURATION.md +++ b/docs/CURATION.md @@ -560,7 +560,7 @@ Ways to get a server behind an endpoint: with `OP_VLM_URL=http://vlm:8000/v1` and the served model name in `OP_VLM_MODEL`. 2. **A server on this host outside compose**: - `OP_VLM_URL=http://host.docker.internal:/v1`. `yolo-api` and the VLM + `OP_VLM_URL=http://host.docker.internal:/v1`. `api` and the VLM worker carry `extra_hosts: host.docker.internal:host-gateway` for this. 3. **A server elsewhere on the network**: `OP_VLM_URL=http://:/v1`. @@ -1189,11 +1189,11 @@ curl -s -X POST $API/train/start -H 'content-type: application/json' -d '{ Nothing here ships a pretrained region detector, VLM or trainer weights. -**Rebuild `yolo-api` before exporting any model.** A pulled image can predate +**Rebuild `api` before exporting any model.** A pulled image can predate the source tree. If `make export-pe` fails with `ModuleNotFoundError: No module named 'core'`, the `perception_models` package -is missing from a stale image: run `docker compose build yolo-api && docker -compose up -d --force-recreate yolo-api`, then re-run the export. +is missing from a stale image: run `docker compose build api && docker +compose up -d --force-recreate api`, then re-run the export. - **PE-Core-L14-336 encoders.** The image tower `pe_image_encoder` is required and its Triton name is fixed (`src/clients/pe_encoder.py`: input @@ -1215,7 +1215,7 @@ compose up -d --force-recreate yolo-api`, then re-run the export. an env var (`OP_INGEST_PRIMARY_CLASS_IDS` is retired and ignored with a warning). To switch detectors set `OP_INGEST_PRIMARY_DETECTOR_MODEL` (and `OP_INGEST_PRIMARY_LABELS_PATH` when - the model directory has no `labels.txt`) and recreate `yolo-api`; the model + the model directory has no `labels.txt`) and recreate `api`; the model must serve the end2end four-tensor output. The choice is deployment-wide. `OP_INGEST_PRIMARY_CONFIDENCE_FLOOR` only applies when the primary assigns classes; a proposer stores every detection the engine emits. A region profile @@ -1301,7 +1301,7 @@ logs one `arbiter_docker_unavailable` warning per outage. ## Mounting your image source Ingest resolves paths under `OP_SOURCE_ROOT` (container default -`/data/source`). Both `yolo-api` and `curation-detection-worker` must mount the +`/data/source`). Both `api` and `curation-detection-worker` must mount the same host directory at the same container path: ```yaml @@ -1309,7 +1309,7 @@ volumes: - ${OP_SOURCE_ROOT_HOST:-./data/source}:/data/source:ro ``` -Set `OP_SOURCE_ROOT_HOST` in `.env`. If only `yolo-api` has the mount, ingest +Set `OP_SOURCE_ROOT_HOST` in `.env`. If only `api` has the mount, ingest succeeds but every later worker read fails with `detection_failed` / `reason=image_unavailable`. `GET /curation/projects/{project}/ingest/config` lists the accepted source roots. @@ -1322,7 +1322,7 @@ Cropwright, or any frontend that consumes `/curation`, needs: | Cropwright env var | Value | Why | |---|---|---| -| `API_UPSTREAM` | `http://op-api:8000` | `yolo-api` has the network alias `op-api`; `http://yolo-api:8000` works too. | +| `API_UPSTREAM` | `http://op-api:8000` | `api` has the network alias `op-api`; `http://api:8000` works too. | | `PUBLIC_API_PREFIX` | `/curation` | Must equal `OP_API_PREFIX`. | | `PUBLIC_TRITON_API_URL` | empty in Docker | The frontend talks to Triton only through the API. | | Docker network | `${COMPOSE_PROJECT_NAME:-openprocessor}_triton_net` | Join it as an external network. | diff --git a/docs/MIGRATION_TRITON_26.md b/docs/MIGRATION_TRITON_26.md index 3117e0d31..b1aadf66f 100644 --- a/docs/MIGRATION_TRITON_26.md +++ b/docs/MIGRATION_TRITON_26.md @@ -15,16 +15,16 @@ on 26.06 (TRT 11.x) with a serialization-version error. docker compose build # Start only what exports need -docker compose up -d triton-server yolo-api +docker compose up -d triton-server api # Re-export every model (runs inside the API container) -docker compose exec yolo-api python /app/export/export_models.py --formats onnx_end2end trt_end2end -docker compose exec yolo-api python /app/export/export_scrfd.py -docker compose exec yolo-api python /app/export/export_face_recognition.py -docker compose exec yolo-api python /app/export/export_mobileclip_image_encoder.py -docker compose exec yolo-api python /app/export/export_mobileclip_text_encoder.py -docker compose exec yolo-api python /app/export/export_paddleocr_det.py -docker compose exec yolo-api python /app/export/export_paddleocr_rec.py +docker compose exec api python /app/export/export_models.py --formats onnx_end2end trt_end2end +docker compose exec api python /app/export/export_scrfd.py +docker compose exec api python /app/export/export_face_recognition.py +docker compose exec api python /app/export/export_mobileclip_image_encoder.py +docker compose exec api python /app/export/export_mobileclip_text_encoder.py +docker compose exec api python /app/export/export_paddleocr_det.py +docker compose exec api python /app/export/export_paddleocr_rec.py # Restart Triton to load the new engines docker compose restart triton-server @@ -76,12 +76,12 @@ automatically, so existing export commands are unchanged. To serve YOLO26 alongside YOLO11: ```bash -docker compose exec yolo-api python /app/export/export_yolo26.py --models small +docker compose exec api python /app/export/export_yolo26.py --models small curl -X POST http://localhost:4603/models/yolo26_small_trt/load curl -F image=@test.jpg 'http://localhost:4603/detect?model_name=yolo26_small_trt' ``` -Set `YOLO_MODEL=yolo26_small_trt` on the yolo-api service to make it the +Set `YOLO_MODEL=yolo26_small_trt` on the api service to make it the default detector. Output-format differences between the families are handled transparently (adapter resolved from Triton model metadata). diff --git a/docs/OCR.md b/docs/OCR.md index a5bb5949c..b051cf0d5 100644 --- a/docs/OCR.md +++ b/docs/OCR.md @@ -120,8 +120,8 @@ The recognition model uses `ppocrv5_dict.txt` with 18383 characters: #### Option A: Use Pre-converted ONNX Models ```bash -# From the yolo-api container: -docker compose exec yolo-api python /app/export/download_paddleocr.py +# From the api container: +docker compose exec api python /app/export/download_paddleocr.py # Or run the convenience script: ./scripts/export_paddleocr.sh download diff --git a/docs/PERFORMANCE.md b/docs/PERFORMANCE.md index 6ffe11fad..a8bf2d583 100644 --- a/docs/PERFORMANCE.md +++ b/docs/PERFORMANCE.md @@ -277,7 +277,7 @@ class TritonConnectionPool: ```bash # Monitor active connections -watch -n 1 'docker compose exec yolo-api netstat -an | grep 8001 | grep ESTABLISHED' +watch -n 1 'docker compose exec api netstat -an | grep 8001 | grep ESTABLISHED' # Monitor latency percentiles # If P99 >1000ms with <5000 RPS = possible connection bottleneck @@ -490,7 +490,7 @@ go run triton_bench.go \ ```bash # Rebuild containers with new requirements docker compose down -docker compose build --no-cache yolo-api +docker compose build --no-cache api docker compose up -d # Wait for warmup (~30 seconds) @@ -560,7 +560,7 @@ py-spy>=0.3.14 Rebuild: ```bash -docker compose build yolo-api +docker compose build api docker compose up -d ``` @@ -626,7 +626,7 @@ Current: **32 workers** (assumes 16-core CPU) 1. Check CPU cores: ```bash -docker exec yolo-api nproc +docker exec api nproc ``` 2. Calculate optimal workers: @@ -641,7 +641,7 @@ Workers = (2 × CPU cores) + 1 4. Restart: ```bash -docker compose restart yolo-api +docker compose restart api ``` **Signs you need fewer workers**: @@ -667,7 +667,7 @@ MAX_FILE_SIZE_MB = 100 # Increase to 100MB Restart: ```bash -docker compose restart yolo-api +docker compose restart api ``` ### Slow Request Threshold @@ -700,10 +700,10 @@ SLOW_REQUEST_THRESHOLD_MS = 50 # More aggressive logging 2. **Not using optimized libraries** ```bash # Verify orjson is installed - docker exec yolo-api python -c "import orjson; print('orjson OK')" + docker exec api python -c "import orjson; print('orjson OK')" # Verify pillow-simd is installed - docker exec yolo-api python -c "from PIL import features; print(features.check_feature('libjpeg_turbo'))" + docker exec api python -c "from PIL import features; print(features.check_feature('libjpeg_turbo'))" ``` ### Issue: Increased Memory Usage @@ -719,7 +719,7 @@ SLOW_REQUEST_THRESHOLD_MS = 50 # More aggressive logging **Monitor**: ```bash # Check memory usage -docker stats yolo-api +docker stats api # Should see periodic drops as workers recycle ``` @@ -730,7 +730,7 @@ docker stats yolo-api 1. Check logs for slow request warnings: ```bash -docker compose logs -f yolo-api | grep "Slow request" +docker compose logs -f api | grep "Slow request" ``` 2. Profile during slow requests: @@ -797,9 +797,9 @@ request_duration = Histogram('api_request_duration_seconds', 'Request duration') 2. Add to Prometheus config: ```yaml -- job_name: 'yolo-api' +- job_name: 'api' static_configs: - - targets: ['yolo-api:4603'] + - targets: ['api:4603'] ``` --- diff --git a/docs/design/README.md b/docs/design/README.md index 029f84cd1..e9cade5df 100644 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -34,7 +34,7 @@ Rows marked implemented are merged for v0.4.0; the rest are open. | #53, #55 | Implemented (shard budget `capacity` block, combine preview warnings) | | #46 | Implemented (region-stage hit-rate gate, pause and resume) | | #61 | VLM proposal noise, optional registry prior, label agreement vs cluster purity | -| #62 | Split oversize files, unify job state files, rename the `yolo-api` compose service | +| #62 | Split oversize files, unify job state files, rename the `api` compose service | | #63 | Release decisions (`:latest`, control-plane-only) and remaining local VLM catalog verification | | #65 | Roadmap: deferred any-domain features | | #38, #39, #41, #42, #43 | Existing follow-ups: image slimming and sizes (#41), 12 GB engine builds (#42), docs screenshots (#43) | diff --git a/docs/design/generic_detector_and_selective_embedding_plan.md b/docs/design/generic_detector_and_selective_embedding_plan.md index 21a53d606..5d9391ece 100644 --- a/docs/design/generic_detector_and_selective_embedding_plan.md +++ b/docs/design/generic_detector_and_selective_embedding_plan.md @@ -69,7 +69,7 @@ F1. Which detector ingest uses, and why it looked vehicle-only. Ingest has NO `_get_ingest_service`, ~L90-L110). The profile is read from env on every call (`src/config/ingest_profiles.py:34` `ingest_primary_profile`), i.e. one detector for the whole deployment, shared by all projects, changed only by - editing `.env` and recreating the `yolo-api` container. The core pipeline + editing `.env` and recreating the `api` container. The core pipeline list loads `yolov11_small_trt_end2end` into Triton (`docker-compose.yml:40`), 80 classes (`models/yolov11_small_trt_end2end/labels.txt`), end2end four-tensor response with max 300 detections and the engine's baked @@ -349,7 +349,7 @@ The ingest walker prints them. - Switching the detector (documented in W1): set `OP_INGEST_PRIMARY_DETECTOR_MODEL` (and `OP_INGEST_PRIMARY_LABELS_PATH` if the - model dir has no `labels.txt`) in `.env`, recreate the `yolo-api` container; + model dir has no `labels.txt`) in `.env`, recreate the `api` container; the model must serve the end2end four-tensor contract (`ingest_detect.py` docstring; a promoted YOLO26 fused output is NOT drop-in, `docs/CURATION.md` L826-L830). It is deployment-wide. Per-project detector @@ -706,7 +706,7 @@ the dirty runtime files in the repo (`models/yolov11_small_trt_end2end/config.pb Baseline (W0, before any code change): with the current stack, throwaway project `gen-base`, ingest the 4,000-image COCO set (manifest from #45; if only 200 are available use those and say so). Run once with the vehicle env set and -once with `OP_INGEST_PRIMARY_CLASS_IDS` unset (recreate `yolo-api`). Record per +once with `OP_INGEST_PRIMARY_CLASS_IDS` unset (recreate `api`). Record per run: wall time, images/s, detections/image, items store bytes (`GET _stats` of the project items index, `size_in_bytes` / docs), images store bytes/image, crop cache growth, embed forwards (Triton metrics), VLM items/s on diff --git a/docs/image_size_audit.md b/docs/image_size_audit.md index 831201fc6..48b9d2b79 100644 --- a/docs/image_size_audit.md +++ b/docs/image_size_audit.md @@ -250,16 +250,16 @@ API container. set with `--target /opt/op-export/site` + a `.pth`, so the layer is the delta, not a second copy of `.local`; `COPY venv-y11`) → `evaluator` (FROM `runtime-base`) → `api` (FROM `runtime-base`, last). -3. **Which service uses which image**: `yolo-api` and all curation workers +3. **Which service uses which image**: `api` and all curation workers → `openprocessor`; new compose service `model-export` (profile `setup`, image `.../openprocessor-export:` (overridable like the other images), same - volumes/GPU reservation as `yolo-api`, no ports) → `openprocessor-export`; + volumes/GPU reservation as `api`, no ports) → `openprocessor-export`; `curation-evaluator` → `openprocessor-evaluator`. 4. **Installer model setup** (`scripts/lib/model_setup.sh`, installer plan - §4.1): replace `dc run --rm --no-deps -T yolo-api python /app/export/...` + §4.1): replace `dc run --rm --no-deps -T api python /app/export/...` with `dc run --rm --no-deps -T model-export python /app/export/...` for groups 0–6 (preflight, weights, YOLO11, MobileCLIP, faces, OCR, PE); - the seed `cp -rn` and the profile-config step can stay on `yolo-api`. + the seed `cp -rn` and the profile-config step can stay on `api`. `export/preflight.py`'s `import tensorrt` check moves to the export image; the API preflight checks `import core` and `torch.cuda` only. Add an export-image override variable to `images.lock`, the `IMAGE_SPECS` table and the diff --git a/env.template b/env.template index 5a92379ef..06bdd3795 100644 --- a/env.template +++ b/env.template @@ -157,7 +157,7 @@ OPENSEARCH_HEAP=2g # existing stack) to bring up a second OpenProcessor on the same host -- # it gets its own container names, network and volumes automatically. # Leaving this unset keeps today's single-stack behavior (project name -# `openprocessor`, container names like `yolo-api`/`triton-server`). +# `openprocessor`, container names like `api`/`triton-server`). # COMPOSE_PROJECT_NAME=opfresh # ============================================================================= @@ -177,7 +177,7 @@ TRITON_GRPC_PORT=4601 TRITON_METRICS_PORT=4602 # GPU id Triton runs on TRITON_GPU_ID=0 -# F-2 (fresh-start E2E findings 2026-09-25): yolo-api (PE/MobileCLIP torch +# F-2 (fresh-start E2E findings 2026-09-25): api (PE/MobileCLIP torch # fallback + CUDA-side preprocessing), curation-evaluator and segmenter # used to hardcode device_ids: ['0'] regardless of where Triton/VLM ran. # On a host whose free GPU isn't 0, change these three (docker-compose.yml @@ -299,8 +299,8 @@ VLM_PORT=4612 # OP_GLOBAL_CONFIGS_INDEX=op_global_configs # Filesystem roots -# G-06: the in-container path yolo-api resolves source-relative item -# paths against -- MUST match the bind target on both yolo-api and +# G-06: the in-container path api resolves source-relative item +# paths against -- MUST match the bind target on both api and # curation-detection-worker (docker-compose.yml mounts # ${OP_SOURCE_ROOT_HOST:-./data/source}:/data/source:ro on both; see # docs/CURATION.md "Mounting your image source"). Set OP_SOURCE_ROOT_HOST @@ -360,7 +360,7 @@ VLM_PORT=4612 # parameters (also served on GET /curation/ingest/config). The streak # (zero_since / how many consecutive zero-reading polls) is persisted to # a small state.json under OP_REGION_DRAIN_STATE_DIR (shared /jobs volume, -# same convention as the job-runner modules) -- multiple yolo-api +# same convention as the job-runner modules) -- multiple api # --workers processes answer this poll at random, so the streak can't # live in module memory or it resets every time a different worker # answers. @@ -391,7 +391,7 @@ VLM_PORT=4612 # examples/region_profiles/ for a worked example) or select a profile a # deployment's own startup code registered by name. G-08: this path is # read INSIDE the container -- docker-compose.yml mounts the repo's -# ./examples at /app/examples:ro on yolo-api and +# ./examples at /app/examples:ro on api and # curation-detection-worker, so a host-relative path like # "examples/region_profiles/..." resolves to nothing in either # container. @@ -550,7 +550,7 @@ VLM_PORT=4612 # ============================================================================= # Combine projects (src/services/projects/combine/, /projects/combine*). # One directory per job (plan, ledger, chunk marks, state) on the shared jobs -# volume, so every yolo-api worker sees it. +# volume, so every api worker sees it. # ============================================================================= # OP_COMBINE_JOBS_DIR=/jobs/combine # OP_COMBINE_PAGE_SIZE=200 @@ -558,7 +558,7 @@ VLM_PORT=4612 # ============================================================================= # C1 active-learning probe job (src/services/curation/probe_job.py, # POST/GET /curation/probe/*). File-backed state.json/heartbeat/cancel.flag -# on the shared /jobs volume -- every yolo-api --workers process reads and +# on the shared /jobs volume -- every api --workers process reads and # writes the same files, so status/cancel/singleton-start are all correct # regardless of which worker process a given request lands on. # ============================================================================= @@ -646,20 +646,20 @@ VLM_PORT=4612 # OP_TRAIN_COMPARE_MAX_IMAGES=200 # OP_TRAIN_COMPARE_INPUT_SIZE=640 # Triton model repository. Read by the trainer (to read an incumbent's -# labels.txt so comparison rows carry names, not bare ids) AND by yolo-api's +# labels.txt so comparison rows carry names, not bare ids) AND by api's # promote path (src/services/training/triton_promote.py) and model discovery # (src/routers/curation/models.py) -- the promote writer and the running # Triton server must agree on this path or a promoted model silently lands # somewhere Triton never sees. Must match whatever host directory is -# bind-mounted into BOTH yolo-api and triton-server; main's own -# docker-compose.yml mounts it at /app/models for yolo-api, so this normally +# bind-mounted into BOTH api and triton-server; main's own +# docker-compose.yml mounts it at /app/models for api, so this normally # doesn't need to be set there -- set it explicitly in any deployment overlay # that mounts the repo somewhere else (e.g. /models). # OP_TRITON_MODEL_REPO=/models # OP_TRITON_HTTP_URL=http://triton-server:8000 # API base the trainer POSTs a campaign's auto-promote to (joined with # OP_API_PREFIX). -# OP_API_BASE_URL=http://yolo-api:8000 +# OP_API_BASE_URL=http://api:8000 # OP_TRAIN_LOG_LEVEL=INFO # MLflow (optional; every hook degrades to a warning when unreachable). # MLFLOW_TRACKING_URI=http://curation-mlflow:5000 @@ -725,7 +725,7 @@ VLM_PORT=4612 # ============================================================================= # Curation export (src/services/curation/export.py) # ============================================================================= -# Baked into yolo-api / curation-trainer / curation-evaluator at BUILD time +# Baked into api / curation-trainer / curation-evaluator at BUILD time # (docker-compose.yml build.args) so every manifest's code_versions.api_sha # and code_versions.trainer_sha reflect the actual built commit, not a dev # checkout's live `git rev-parse HEAD` fallback (`make build`/`make rebuild` @@ -762,7 +762,7 @@ VLM_PORT=4612 # Default BakeoffProfile when a job names none: a registered name (generic) # or a path to a profile .json file. Unset = generic (scores every class in # the test split). Example profiles are opt-in by path and need examples/ -# mounted into both yolo-api and the evaluator, e.g. with ./examples mounted +# mounted into both api and the evaluator, e.g. with ./examples mounted # at /app/examples: # OP_BAKEOFF_PROFILE=/app/examples/bakeoff/license_plate/profile.json # OP_BAKEOFF_PROFILE= @@ -829,7 +829,7 @@ VLM_PORT=4612 # then OP_VLM_URL=http://vlm:8000/v1 (below). See "In-compose VLM # (--profile vlm)" further down for its own knobs. # 2. Point at one already running on THIS host (outside compose): -# OP_VLM_URL=http://host.docker.internal:/v1 -- yolo-api and +# OP_VLM_URL=http://host.docker.internal:/v1 -- api and # curation-vlm-worker both carry the `host.docker.internal:host-gateway` # extra_hosts entry needed for this to resolve on Linux (Docker # Desktop adds it for you; the Linux engine does not by default). diff --git a/export/README.md b/export/README.md index e407c8029..bf04acbc2 100644 --- a/export/README.md +++ b/export/README.md @@ -89,7 +89,7 @@ models/ make export-models # Or directly: -docker compose exec yolo-api python /app/export/export_models.py \ +docker compose exec api python /app/export/export_models.py \ --models small \ --formats trt trt_end2end \ --normalize-boxes @@ -114,18 +114,18 @@ output: ```bash # ONNX only (CPU-friendly; validates via onnxruntime round-trip) -docker compose exec yolo-api python /app/export/export_detector_dual_head.py \ +docker compose exec api python /app/export/export_detector_dual_head.py \ --weights /app/pytorch_models/my_detector.pt \ --triton-name my_detector_dual_head --imgsz 640 # ONNX + TensorRT engine + config.pbtxt + labels.txt into the model repo -docker compose exec yolo-api python /app/export/export_detector_dual_head.py \ +docker compose exec api python /app/export/export_detector_dual_head.py \ --weights /app/pytorch_models/my_detector.pt \ --triton-name my_detector_dual_head \ --imgsz 1280 --max-batch 16 --formats onnx trt # Legacy YOLOv5-fork checkpoint (the fork is only needed to load its checkpoints) -docker compose exec yolo-api python /app/export/export_detector_dual_head.py \ +docker compose exec api python /app/export/export_detector_dual_head.py \ --weights /app/pytorch_models/legacy_v5.pt --loader yolov5 \ --yolov5-fork /app/external/yolov5 \ --imgsz 1280 --triton-name legacy_v5_dual_head @@ -155,17 +155,17 @@ engine produces it. `--formats trt` writes that file for you. ### SCRFD Face Detection ```bash -docker compose exec yolo-api python /app/export/export_scrfd.py +docker compose exec api python /app/export/export_scrfd.py ``` ### Face Recognition (ArcFace) ```bash # Download pre-trained model -docker compose exec yolo-api python /app/export/download_face_models.py +docker compose exec api python /app/export/download_face_models.py # Export to TensorRT -docker compose exec yolo-api python /app/export/export_face_recognition.py +docker compose exec api python /app/export/export_face_recognition.py ``` ### MobileCLIP (Visual Search) @@ -175,8 +175,8 @@ docker compose exec yolo-api python /app/export/export_face_recognition.py make export-mobileclip # Or individually: -docker compose exec yolo-api python /app/export/export_mobileclip_image_encoder.py -docker compose exec yolo-api python /app/export/export_mobileclip_text_encoder.py +docker compose exec api python /app/export/export_mobileclip_image_encoder.py +docker compose exec api python /app/export/export_mobileclip_text_encoder.py ``` ### PE-Core Encoders (Curation Embeddings) @@ -193,7 +193,7 @@ Both embeddings come out L2-normalized. See [`docs/CURATION.md`](../docs/CURATION.md#models-you-must-supply) for what depends on them. -**Fresh deployment, end to end** (API container = `yolo-api`; it has torch, +**Fresh deployment, end to end** (API container = `api`; it has torch, `perception_models` and the HF cache mounted from `./cache/huggingface`): ```bash @@ -234,7 +234,7 @@ The ONNX export runs in the API container, the TensorRT build in the Triton image (that's where `trtexec` lives): ```bash -docker compose exec yolo-api python /app/export/export_pe_image_encoder.py +docker compose exec api python /app/export/export_pe_image_encoder.py ONNX_PATH=./pytorch_models/pe_image_encoder.onnx bash export/build_pe_trt.sh # Path 1 ONNX_PATH=./pytorch_models/pe_image_encoder.onnx bash export/build_pe_ort_fallback.sh # Path 2 ``` @@ -264,7 +264,7 @@ is the rendered default (TensorRT, max batch 32). #### 3. Text tower — `export_pe_text_encoder.py` ```bash -docker compose exec yolo-api python /app/export/export_pe_text_encoder.py [--benchmark] +docker compose exec api python /app/export/export_pe_text_encoder.py [--benchmark] ``` Exports `clip.encode_text(tokens, normalize=True)` — causal transformer, @@ -324,7 +324,7 @@ Optional Triton serving (onnxruntime backend, CPU instances by default so it takes no VRAM; `--kind gpu --gpus N` for GPU): ```bash -docker compose exec yolo-api python /app/export/export_pe_text_encoder.py \ +docker compose exec api python /app/export/export_pe_text_encoder.py \ --install-triton --models-dir /app/models # = make pe-export-text-triton # pe_text_encoder is already in the default --load-model list of docker-compose.yml ``` @@ -367,11 +367,11 @@ configured or `OP_PE_TEXT_BACKEND=triton`. ```bash # Download PP-OCRv5 models -docker compose exec yolo-api python /app/export/download_paddleocr.py +docker compose exec api python /app/export/download_paddleocr.py # Export detection and recognition -docker compose exec yolo-api python /app/export/export_paddleocr_det.py -docker compose exec yolo-api python /app/export/export_paddleocr_rec.py +docker compose exec api python /app/export/export_paddleocr_det.py +docker compose exec api python /app/export/export_paddleocr_rec.py ``` ## Model Specifications @@ -447,7 +447,7 @@ make download-models ### "Failed to build TensorRT engine" Check GPU memory and reduce batch size if needed: ```bash -docker compose exec yolo-api nvidia-smi +docker compose exec api nvidia-smi ``` ### Triton fails to load model diff --git a/export/download_face_models.py b/export/download_face_models.py index 01cb4b637..79ac7d546 100755 --- a/export/download_face_models.py +++ b/export/download_face_models.py @@ -11,8 +11,8 @@ - InsightFace HuggingFace model hub Usage: - # From yolo-api container: - docker compose exec yolo-api python /app/export/download_face_models.py + # From api container: + docker compose exec api python /app/export/download_face_models.py # Or from host with venv: python export/download_face_models.py diff --git a/export/download_paddleocr.py b/export/download_paddleocr.py index 6e5010eb6..5f707e8b9 100755 --- a/export/download_paddleocr.py +++ b/export/download_paddleocr.py @@ -12,8 +12,8 @@ - RapidOCR model repository Usage: - # From yolo-api container: - docker compose exec yolo-api python /app/export/download_paddleocr.py + # From api container: + docker compose exec api python /app/export/download_paddleocr.py # Or from host with venv: python export/download_paddleocr.py diff --git a/export/download_pe_weights.py b/export/download_pe_weights.py index ead732c30..f281ad67b 100755 --- a/export/download_pe_weights.py +++ b/export/download_pe_weights.py @@ -40,7 +40,7 @@ Usage ----- # In the API container (recommended; the cache is mounted there) - docker compose exec yolo-api python /app/export/download_pe_weights.py + docker compose exec api python /app/export/download_pe_weights.py # Anywhere with huggingface_hub installed; print only the resolved path python export/download_pe_weights.py --print-path diff --git a/export/export_detector_dual_head.py b/export/export_detector_dual_head.py index 8faed7cdf..62e58f198 100755 --- a/export/export_detector_dual_head.py +++ b/export/export_detector_dual_head.py @@ -41,18 +41,18 @@ Usage ----- # ONNX only (no GPU needed) - docker compose exec yolo-api python /app/export/export_detector_dual_head.py \\ + docker compose exec api python /app/export/export_detector_dual_head.py \\ --weights /app/pytorch_models/my_detector.pt \\ --triton-name my_detector_dual_head # ONNX + TensorRT engine + config.pbtxt written into the model repo - docker compose exec yolo-api python /app/export/export_detector_dual_head.py \\ + docker compose exec api python /app/export/export_detector_dual_head.py \\ --weights /app/pytorch_models/my_detector.pt \\ --triton-name my_detector_dual_head \\ --imgsz 1280 --max-batch 16 --formats onnx trt # A YOLOv5-fork checkpoint - docker compose exec yolo-api python /app/export/export_detector_dual_head.py \\ + docker compose exec api python /app/export/export_detector_dual_head.py \\ --weights /app/pytorch_models/legacy_v5.pt --loader yolov5 \\ --yolov5-fork /app/external/yolov5 --imgsz 1280 \\ --triton-name legacy_v5_dual_head diff --git a/export/export_face_recognition.py b/export/export_face_recognition.py index 7cd578086..0fd067f8e 100755 --- a/export/export_face_recognition.py +++ b/export/export_face_recognition.py @@ -19,8 +19,8 @@ # From host with venv: python export/export_face_recognition.py - # From yolo-api container: - docker compose exec yolo-api python /app/export/export_face_recognition.py + # From api container: + docker compose exec api python /app/export/export_face_recognition.py """ import argparse diff --git a/export/export_mobileclip_image_encoder.py b/export/export_mobileclip_image_encoder.py index 3a99f3aaa..7ba37ef1e 100755 --- a/export/export_mobileclip_image_encoder.py +++ b/export/export_mobileclip_image_encoder.py @@ -18,9 +18,9 @@ 4. Convert to TensorRT plan for maximum throughput 5. Validate output matches PyTorch -Run from: yolo-api container - docker compose exec yolo-api python /app/export/export_mobileclip_image_encoder.py --model S2 - docker compose exec yolo-api python /app/export/export_mobileclip_image_encoder.py --model B +Run from: api container + docker compose exec api python /app/export/export_mobileclip_image_encoder.py --model S2 + docker compose exec api python /app/export/export_mobileclip_image_encoder.py --model B """ import argparse diff --git a/export/export_mobileclip_text_encoder.py b/export/export_mobileclip_text_encoder.py index e3fd5b3a1..1c5821eb7 100755 --- a/export/export_mobileclip_text_encoder.py +++ b/export/export_mobileclip_text_encoder.py @@ -16,8 +16,8 @@ 4. Convert to TensorRT plan for maximum throughput 5. Validate output matches PyTorch -Run from: yolo-api container - docker compose exec yolo-api python /app/export/export_mobileclip_text_encoder.py --model S2 +Run from: api container + docker compose exec api python /app/export/export_mobileclip_text_encoder.py --model S2 """ import argparse diff --git a/export/export_models.py b/export/export_models.py index 4ef750089..f6e61e070 100755 --- a/export/export_models.py +++ b/export/export_models.py @@ -16,16 +16,16 @@ Usage: ------ # Export all formats (default) -docker compose exec yolo-api python /app/export/export_models.py +docker compose exec api python /app/export/export_models.py # Export only end2end models -docker compose exec yolo-api python /app/export/export_models.py --formats onnx_end2end trt_end2end +docker compose exec api python /app/export/export_models.py --formats onnx_end2end trt_end2end # Export specific models -docker compose exec yolo-api python /app/export/export_models.py --models nano small +docker compose exec api python /app/export/export_models.py --models nano small # Export everything -docker compose exec yolo-api python /app/export/export_models.py --formats all +docker compose exec api python /app/export/export_models.py --formats all """ import os diff --git a/export/export_paddleocr_det.py b/export/export_paddleocr_det.py index 703b39e98..b37e14ad1 100755 --- a/export/export_paddleocr_det.py +++ b/export/export_paddleocr_det.py @@ -10,8 +10,8 @@ - Output: [B, 1, H, W] probability map for text regions - Preprocessing: (x / 255 - 0.5) / 0.5 = x / 127.5 - 1, BGR format -Run from: yolo-api container - docker compose exec yolo-api python /app/export/export_paddleocr_det.py +Run from: api container + docker compose exec api python /app/export/export_paddleocr_det.py """ import argparse @@ -310,7 +310,7 @@ def main(): if not args.onnx_path.exists(): print(f'\nERROR: ONNX model not found: {args.onnx_path}') print('\nTo download, run:') - print(' docker compose exec yolo-api python /app/export/download_paddleocr.py') + print(' docker compose exec api python /app/export/download_paddleocr.py') return 1 # Verify ONNX diff --git a/export/export_paddleocr_rec.py b/export/export_paddleocr_rec.py index 66fe33546..6aaa7a922 100755 --- a/export/export_paddleocr_rec.py +++ b/export/export_paddleocr_rec.py @@ -187,13 +187,13 @@ def convert_to_tensorrt_via_python_api(onnx_path: Path, plan_path: Path) -> Path """Build the TensorRT engine in-process with the TensorRT Python API. F-17 (fresh-start E2E findings 2026-09-25): this script runs inside - ``yolo-api`` (see the module docstring's ``Usage``), which has no + ``api`` (see the module docstring's ``Usage``), which has no docker CLI or docker.sock, and shelling out to ``docker exec trtexec ...`` also hardcoded the pre-G-01 container name ``triton-server`` -- after the project-name rename it's ``${COMPOSE_PROJECT_NAME}-triton``, so this always printed "triton-server container is not running" even with Triton - up. yolo-api already depends on ``tensorrt-cu13`` directly (see + up. api already depends on ``tensorrt-cu13`` directly (see requirements.txt; ``export/export_models.py`` builds YOLO engines the same way), so there's no need for Triton to be running, or even exist, at export time -- only the ONNX file and a GPU. diff --git a/export/export_pe_image_encoder.py b/export/export_pe_image_encoder.py index d9100d2a0..ab6cb0dea 100755 --- a/export/export_pe_image_encoder.py +++ b/export/export_pe_image_encoder.py @@ -59,7 +59,7 @@ Usage ----- # ONNX + config.pbtxt (default: perception-models method) - docker compose exec yolo-api python /app/export/export_pe_image_encoder.py + docker compose exec api python /app/export/export_pe_image_encoder.py # ONNX only, custom destination, no config.pbtxt python export/export_pe_image_encoder.py \\ diff --git a/export/export_pe_text_encoder.py b/export/export_pe_text_encoder.py index f9a5f8511..9e889c2cb 100755 --- a/export/export_pe_text_encoder.py +++ b/export/export_pe_text_encoder.py @@ -63,10 +63,10 @@ Usage ----- # ONNX + parity gate (API container: has torch + perception_models) - docker compose exec yolo-api python /app/export/export_pe_text_encoder.py + docker compose exec api python /app/export/export_pe_text_encoder.py # Also install as a Triton model - docker compose exec yolo-api python /app/export/export_pe_text_encoder.py \\ + docker compose exec api python /app/export/export_pe_text_encoder.py \\ --install-triton --models-dir /app/models # Re-render only the Triton config.pbtxt (no torch needed) diff --git a/export/export_scrfd.py b/export/export_scrfd.py index 1282d94f3..3f831108c 100755 --- a/export/export_scrfd.py +++ b/export/export_scrfd.py @@ -22,8 +22,8 @@ 5. Create Triton configuration Usage: - # From yolo-api container: - docker compose exec yolo-api python /app/export/export_scrfd.py + # From api container: + docker compose exec api python /app/export/export_scrfd.py # From host with venv: python export/export_scrfd.py @@ -510,7 +510,7 @@ def convert_to_tensorrt( return True except ImportError: - logger.error('TensorRT not available. Run inside yolo-api container.') + logger.error('TensorRT not available. Run inside api container.') return False except Exception as e: logger.error(f'TensorRT conversion failed: {e}') diff --git a/export/export_yolo26.py b/export/export_yolo26.py index beffadd28..edd4c4ab1 100755 --- a/export/export_yolo26.py +++ b/export/export_yolo26.py @@ -19,10 +19,10 @@ Usage: ------ # Export nano + small to TensorRT (default) -docker compose exec yolo-api python /app/export/export_yolo26.py +docker compose exec api python /app/export/export_yolo26.py # Specific sizes / formats -docker compose exec yolo-api python /app/export/export_yolo26.py --models small --formats trt +docker compose exec api python /app/export/export_yolo26.py --models small --formats trt """ import argparse diff --git a/export/models_config.example.yaml b/export/models_config.example.yaml index 4cf3baf46..48793a371 100644 --- a/export/models_config.example.yaml +++ b/export/models_config.example.yaml @@ -7,7 +7,7 @@ # Usage: # make export-config CONFIG=/app/export/models_config.yaml # # or -# docker compose exec yolo-api python /app/export/export_models.py \ +# docker compose exec api python /app/export/export_models.py \ # --config-file /app/export/models_config.yaml \ # --formats trt trt_end2end \ # --normalize-boxes \ diff --git a/export/preflight.py b/export/preflight.py index 3e10674fb..ae21e8345 100644 --- a/export/preflight.py +++ b/export/preflight.py @@ -4,7 +4,7 @@ section 3.2): a stale ``:latest`` API image once shipped without ``perception_models`` importable. This script is invoked as - docker compose run --rm --no-deps -T yolo-api python -m export.preflight + docker compose run --rm --no-deps -T api python -m export.preflight and exits non-zero with a specific message per failing check, instead of failing deep inside the first export step with a confusing traceback. diff --git a/monitoring/README.md b/monitoring/README.md index ad3af4fe2..5908e6485 100644 --- a/monitoring/README.md +++ b/monitoring/README.md @@ -32,7 +32,7 @@ docker compose logs -f grafana | Loki | http://localhost:4606 | - | | Triton Metrics | http://localhost:4602/metrics | - | -The API job (`yolo-api:8000/metrics`) is scraped once and already aggregates every API worker (multiprocess mode, `PROMETHEUS_MULTIPROC_DIR` on a tmpfs in compose). +The API job (`api:8000/metrics`) is scraped once and already aggregates every API worker (multiprocess mode, `PROMETHEUS_MULTIPROC_DIR` on a tmpfs in compose). ## Dashboard @@ -173,7 +173,7 @@ grafana: {container="/triton-server"} # FastAPI logs -{container=~"/(yolo-api|pytorch-api)"} +{container=~"/(api|pytorch-api)"} # Error logs only {container="/triton-server"} |= "error" or "ERROR" diff --git a/monitoring/alloy-config.alloy b/monitoring/alloy-config.alloy index 2e7d4c62a..f54ca391d 100644 --- a/monitoring/alloy-config.alloy +++ b/monitoring/alloy-config.alloy @@ -55,7 +55,7 @@ discovery.relabel "fastapi" { } rule { source_labels = ["__meta_docker_container_label_com_docker_compose_service"] - regex = "yolo-api" + regex = "api" action = "keep" } rule { diff --git a/monitoring/dashboards/overview.json b/monitoring/dashboards/overview.json index 5def8a3c2..aed47ef1b 100644 --- a/monitoring/dashboards/overview.json +++ b/monitoring/dashboards/overview.json @@ -115,7 +115,7 @@ "type": "prometheus", "uid": "${ds}" }, - "expr": "avg(up{job=~\"triton|yolo-api|dcgm|node\"})", + "expr": "avg(up{job=~\"triton|api|dcgm|node\"})", "legendFormat": "", "refId": "A", "instant": false @@ -507,7 +507,7 @@ "type": "prometheus", "uid": "${ds}" }, - "expr": "up{job=~\"triton|yolo-api|dcgm|node|loki\"}", + "expr": "up{job=~\"triton|api|dcgm|node|loki\"}", "legendFormat": "{{job}}", "refId": "A" } diff --git a/monitoring/prometheus.yml b/monitoring/prometheus.yml index 408a64f2e..d626e98c4 100644 --- a/monitoring/prometheus.yml +++ b/monitoring/prometheus.yml @@ -13,10 +13,10 @@ scrape_configs: labels: group: 'triton_inference_server' - - job_name: 'yolo-api' + - job_name: 'api' metrics_path: /metrics static_configs: - - targets: ['yolo-api:8000'] + - targets: ['api:8000'] labels: group: 'api' diff --git a/openprocessor b/openprocessor index f307da37c..7ac5e488b 100755 --- a/openprocessor +++ b/openprocessor @@ -198,7 +198,7 @@ cmd_start() { log_step "Starting OpenProcessor services..." cd "$PROJECT_DIR" ensure_bind_mount_dirs "$PROJECT_DIR" - dc up -d + dc up -d --remove-orphans log_success "Services started" echo "" cmd_status @@ -389,7 +389,7 @@ cmd_models_install() { return 1 fi if [[ -z "$only" || "$only" == pe ]] && [[ "$tiers" == *curation* ]]; then - dc restart yolo-api + dc restart api fi } @@ -733,7 +733,7 @@ Setup: Examples: ./scripts/openprocessor.sh status - ./scripts/openprocessor.sh logs yolo-api -f + ./scripts/openprocessor.sh logs api -f ./scripts/openprocessor.sh export all ./scripts/openprocessor.sh profile standard ./scripts/openprocessor.sh test full diff --git a/scripts/README.md b/scripts/README.md index 4c2f8ae6b..559f5ca9f 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -4,7 +4,7 @@ Setup and management scripts, release tooling, dataset fetchers, code generators and the curation workers and tools. Run Python scripts with the project venv binary (`.venv/bin/python`), or inside -the `yolo-api` container where the script says so. +the `api` container where the script says so. ## Root-level scripts @@ -24,8 +24,8 @@ most: ```bash ./openprocessor status # containers, health, GPU memory -./openprocessor logs yolo-api -f # follow one service -./openprocessor restart yolo-api +./openprocessor logs api -f # follow one service +./openprocessor restart api ./openprocessor models # Triton model states ./openprocessor models install --only pe # export and load one model group ./openprocessor curation up # start the curation workers @@ -147,7 +147,7 @@ before touching OpenSearch. | Path | What it is | |---|---| -| `ingest_walker.py`, `_fast_walk.py` | Parallel bulk-directory ingest. Walks a directory the API container can see and posts paths to `POST /curation/projects/{project}/ingest/batch`, with a resumable progress file. Run it inside `yolo-api`: `--root /data/source/...` | +| `ingest_walker.py`, `_fast_walk.py` | Parallel bulk-directory ingest. Walks a directory the API container can see and posts paths to `POST /curation/projects/{project}/ingest/batch`, with a resumable progress file. Run it inside `api`: `--root /data/source/...` | | `ingest_upload.py` | Byte-upload ingest for storage the API cannot mount: reads files locally and posts them to `POST /curation/projects/{project}/ingest/upload`; resumes with `POST /curation/projects/{project}/ingest/path_lookup` and server-side content-hash dedup | | `import_labeled_dataset.py` | Client of `POST /curation/projects/{project}/datasets/imports`. Previews a dataset (YOLO, COCO, OpenProcessor export), builds the by-name mapping from `--map CLASS=ID`, `--create CLASS[=NAME]`, `--skip`, `--region`, `--accept-suggestions` (`--images-only` skips every class), starts or resumes (`--resume IMPORT_ID`) and polls. `--dry-run` previews only; `--state-dir` writes the per-split cohort `eval_regions_vs_gt.py` reads | | `yolo_dataset.py` | Shared YOLO dataset discovery for the two tools above | diff --git a/scripts/curation/auto_label_worker.py b/scripts/curation/auto_label_worker.py index 5552a9000..01a96bd01 100755 --- a/scripts/curation/auto_label_worker.py +++ b/scripts/curation/auto_label_worker.py @@ -3,7 +3,7 @@ Replaces the per-request ``subprocess.Popen`` model in :mod:`src.services.curation.autolabel.job` with a long-lived worker -process. The yolo-api just drops a JSON trigger file under +process. The api just drops a JSON trigger file under ``/trigger.json`` and returns 202; this worker picks it up, runs the pipeline, and writes state.json updates the SSE endpoint already streams to the dashboard. @@ -14,7 +14,7 @@ OpenSearch error lands in ``docker compose logs curation-auto-label-worker`` — no more 'subprocess vanished before writing terminal state' opacity. -* **Survives API restarts**: stop/rm/up yolo-api does not kill an +* **Survives API restarts**: stop/rm/up api does not kill an in-flight recluster; the worker keeps running. * **Lifecycle ownership**: docker compose owns restart-on-crash via ``restart: unless-stopped`` — the API no longer races to detect a @@ -31,7 +31,7 @@ ``{job_id, pipeline, args}`` written by ``start_job``. Worker claims it via ``unlink`` (atomic on Linux) then runs the pipeline. * State file ``state.json`` — worker writes this as it advances - stages; each project's SSE stream (yolo-api) polls its mtime. + stages; each project's SSE stream (api) polls its mtime. * Heartbeat file ``heartbeat`` — worker touches every 5 s while running; API uses mtime to detect a dead worker. * Cancel flag ``cancel.flag`` — operator writes via diff --git a/scripts/curation/bakeoff/requirements.txt b/scripts/curation/bakeoff/requirements.txt index a229117ca..439fbd676 100644 --- a/scripts/curation/bakeoff/requirements.txt +++ b/scripts/curation/bakeoff/requirements.txt @@ -2,7 +2,7 @@ # # The harness must run in a YOLO26-capable container (the a YOLO-training-capable # image installs Ultralytics from a pinned git SHA + torch + GPU + mlflow -# 2.18 matching the MLflow tracking server). It must NOT run in yolo-api, +# 2.18 matching the MLflow tracking server). It must NOT run in api, # which pins `ultralytics<8.4` and therefore cannot load a YOLO26 model. # # These are the packages the trainer image does NOT already have. Add them diff --git a/scripts/curation/vlm_worker.py b/scripts/curation/vlm_worker.py index 2e515e9ce..d3c6cdeac 100755 --- a/scripts/curation/vlm_worker.py +++ b/scripts/curation/vlm_worker.py @@ -29,7 +29,7 @@ Or as the long-lived compose service (G-10: there is no ``make curation-vlm-worker`` target -- use one of these instead): docker compose --profile curation up -d curation-vlm-worker - docker compose exec yolo-api python scripts/curation/vlm_worker.py --until-empty + docker compose exec api python scripts/curation/vlm_worker.py --until-empty """ from __future__ import annotations diff --git a/scripts/datasets/fetch_coco_subset.py b/scripts/datasets/fetch_coco_subset.py index 96595c684..1c000daf9 100755 --- a/scripts/datasets/fetch_coco_subset.py +++ b/scripts/datasets/fetch_coco_subset.py @@ -33,7 +33,7 @@ ``make sample-coco-readme``, or directly with the project venv), never via ``docker compose exec``. ``--out`` is resolved relative to the current working directory, and ``docker-compose.yml`` mounts -``${OP_SOURCE_ROOT_HOST:-./data/source}`` into the ``yolo-api`` / +``${OP_SOURCE_ROOT_HOST:-./data/source}`` into the ``api`` / ``curation-detection-worker`` containers **read-only** at ``/data/source``: a container path under that mount raises ``OSError: Read-only filesystem``. The host filesystem itself has no diff --git a/scripts/docker-build-push.sh b/scripts/docker-build-push.sh index dddcf98c3..21ba3802e 100755 --- a/scripts/docker-build-push.sh +++ b/scripts/docker-build-push.sh @@ -145,7 +145,7 @@ run_security_scan() { # ============================================================================= build_api() { - print_header "Building API Image (yolo-api)" + print_header "Building API Image (api)" print_info "Platforms: ${PLATFORMS}" print_info "Version: ${VERSION_FULL}" print_info "Tags: latest, ${VERSION_FULL}" @@ -311,7 +311,7 @@ Usage: $0 [OPTION] Build and push Docker images to Docker Hub for OpenProcessor Options: - api Build and push only API image (yolo-api/FastAPI) + api Build and push only API image (api/FastAPI) triton Build and push only Triton image (inference server) all Build and push both images (default) local Build locally without pushing diff --git a/scripts/export_paddleocr.sh b/scripts/export_paddleocr.sh index 57a20f666..1de905566 100755 --- a/scripts/export_paddleocr.sh +++ b/scripts/export_paddleocr.sh @@ -161,14 +161,14 @@ download_models() { # Fall back to Docker container if host .venv didn't work if [ "$download_ok" = false ]; then - if check_container "yolo-api"; then - log_info "Using running yolo-api container for download..." - if docker compose exec -T yolo-api python /app/export/download_paddleocr.py; then + if check_container "api"; then + log_info "Using running api container for download..." + if docker compose exec -T api python /app/export/download_paddleocr.py; then download_ok=true fi else - log_info "Using temporary yolo-api container for download..." - if docker compose run --rm --no-deps -T yolo-api python /app/export/download_paddleocr.py; then + log_info "Using temporary api container for download..." + if docker compose run --rm --no-deps -T api python /app/export/download_paddleocr.py; then download_ok=true fi fi @@ -178,7 +178,7 @@ download_models() { log_success "ONNX models downloaded (detection + multilingual recognition)" else log_error "Model download failed" - log_info "Ensure either .venv or yolo-api container image is built" + log_info "Ensure either .venv or api container image is built" return 1 fi } diff --git a/scripts/lib/config.sh b/scripts/lib/config.sh index 7c45b35a4..73a373738 100755 --- a/scripts/lib/config.sh +++ b/scripts/lib/config.sh @@ -768,7 +768,7 @@ services: capabilities: - gpu - yolo-api: + api: shm_size: ${PROFILE_SHM_SIZE} command: - uvicorn diff --git a/scripts/lib/download.sh b/scripts/lib/download.sh index b6722972f..eadf356ba 100755 --- a/scripts/lib/download.sh +++ b/scripts/lib/download.sh @@ -279,13 +279,13 @@ download_paddleocr_models() { fi # Try running container, fall back to temporary container - if docker ps --format '{{.Names}}' | grep -q "^yolo-api$"; then - if dc exec -T yolo-api python /app/export/download_paddleocr.py; then + if docker ps --format '{{.Names}}' | grep -q "^api$"; then + if dc exec -T api python /app/export/download_paddleocr.py; then log_success "PaddleOCR models downloaded" return 0 fi else - if dc run --rm --no-deps -T yolo-api python /app/export/download_paddleocr.py; then + if dc run --rm --no-deps -T api python /app/export/download_paddleocr.py; then log_success "PaddleOCR models downloaded" return 0 fi diff --git a/scripts/lib/export.sh b/scripts/lib/export.sh index 57309c8e1..dfb6e4600 100755 --- a/scripts/lib/export.sh +++ b/scripts/lib/export.sh @@ -49,11 +49,11 @@ declare -gA EXPORT_TIMES=( # Container Helpers # ============================================================================= -# Run a command in a temporary yolo-api container (no dependencies needed). +# Run a command in a temporary api container (no dependencies needed). # Uses 'docker compose run --rm --no-deps -T' so the export works without -# Triton running and without the yolo-api container already up. +# Triton running and without the api container already up. run_in_api_container() { - dc run --rm --no-deps -T yolo-api "$@" + dc run --rm --no-deps -T api "$@" } check_triton_container() { diff --git a/scripts/lib/model_setup.sh b/scripts/lib/model_setup.sh index 1169a463c..192556c70 100755 --- a/scripts/lib/model_setup.sh +++ b/scripts/lib/model_setup.sh @@ -263,7 +263,7 @@ model_setup_group_outputs() { } # One-shot API-image and Triton-image containers (never need a running API). -_ms_api() { dc run --rm --no-deps -T -w /app yolo-api "$@"; } +_ms_api() { dc run --rm --no-deps -T -w /app api "$@"; } _ms_triton() { dc run --rm --no-deps -T triton-server "$@"; } _ms_pe_trtexec() { @@ -306,7 +306,7 @@ _ms_run_group_steps() { case "$group" in preflight) "$runner" preflight_image _ms_api python -m export.preflight || return 1 - "$runner" preflight_seed dc run --rm --no-deps -T --entrypoint cp yolo-api \ + "$runner" preflight_seed dc run --rm --no-deps -T --entrypoint cp api \ -rn /opt/openprocessor/model_repo_seed/. /app/models/ || return 1 "$runner" preflight_profile _ms_api bash -c \ 'PROJECT_DIR=/app; source /app/scripts/lib/config.sh && generate_all_configs "$1" 0' \ diff --git a/scripts/lib/vlm_switch.sh b/scripts/lib/vlm_switch.sh index 5b042a3d5..5e898c0b7 100755 --- a/scripts/lib/vlm_switch.sh +++ b/scripts/lib/vlm_switch.sh @@ -122,7 +122,7 @@ _vlm_own_mib() { # _vlm_in_api SUBCOMMAND -> runs one of the three fixed state-volume actions # inside the API container (the state volume is not visible from the host) _vlm_in_api() { - dc exec -T yolo-api sh -c ' + dc exec -T api sh -c ' d="'"${_VLM_STATE_IN_CONTAINER}"'" s="$d/vlm_worker/pause.sentinel" case "$1" in @@ -185,7 +185,7 @@ _vlm_recreate_targets() { echo vlm [[ "$clients" == 1 ]] || return 0 running="$(dc ps --services --status running 2>/dev/null || true)" - for svc in yolo-api curation-detection-worker curation-vlm-worker curation-auto-label-worker; do + for svc in api curation-detection-worker curation-vlm-worker curation-auto-label-worker; do [[ $'\n'"$running"$'\n' == *$'\n'"$svc"$'\n'* ]] && echo "$svc" done return 0 @@ -349,7 +349,7 @@ _vlm_switch() { local created_pause=0 paused paused="$(_vlm_in_api pause-create 2>/dev/null || true)" if [[ "$paused" != created && "$paused" != present ]]; then - log_error "cannot reach the yolo-api container to pause the workers; is the stack running?" + log_error "cannot reach the api container to pause the workers; is the stack running?" return 1 fi [[ "$paused" == created ]] && created_pause=1 diff --git a/scripts/security-scan.sh b/scripts/security-scan.sh index 72cc1c643..9928745d3 100755 --- a/scripts/security-scan.sh +++ b/scripts/security-scan.sh @@ -458,7 +458,7 @@ Tools used: - Grype: Fast vulnerability scanner Options: - api Scan only API image (yolo-api/FastAPI) + api Scan only API image (api/FastAPI) triton Scan only Triton image (inference server) all Scan both images (default) install Install all required tools diff --git a/scripts/setup.sh b/scripts/setup.sh index f9de7df7b..ada2a8f88 100755 --- a/scripts/setup.sh +++ b/scripts/setup.sh @@ -414,7 +414,7 @@ build_docker_images() { # Fall back to building from Dockerfiles if pull fails (e.g., first release, # no internet, or user wants a custom build). log_step "Pulling pre-built images from Docker Hub..." - if dc pull yolo-api triton-server 2>/dev/null; then + if dc pull api triton-server 2>/dev/null; then log_success "Pre-built images pulled from Docker Hub" return 0 else @@ -424,13 +424,13 @@ build_docker_images() { fi # Build from Dockerfiles - log_step "Building yolo-api image..." - if ! dc build yolo-api; then - log_error "Failed to build yolo-api image" + log_step "Building api image..." + if ! dc build api; then + log_error "Failed to build api image" log_info "Check your internet connection and disk space (need ~5GB free)" return 1 fi - log_success "yolo-api image ready" + log_success "api image ready" log_step "Building triton-server image..." if ! dc build triton-server; then @@ -485,7 +485,7 @@ export_models_step() { # All exports use 'docker compose run --rm --no-deps -T' to create temporary # containers. This avoids the chicken-and-egg problem where triton-server - # crashes because model.plan files don't exist yet, and yolo-api crash-loops + # crashes because model.plan files don't exist yet, and api crash-loops # because it can't connect to triton-server. # Export based on profile @@ -559,7 +559,7 @@ start_services_step() { # Start all services (now that model.plan files exist, triton-server will load them) log_info "Starting all services..." - dc up -d + dc up -d --remove-orphans # Wait for services to be ready wait_for_services diff --git a/setup-openprocessor.sh b/setup-openprocessor.sh index d1067b506..ef7607e35 100755 --- a/setup-openprocessor.sh +++ b/setup-openprocessor.sh @@ -1938,14 +1938,14 @@ run_health() { if [[ "$OP_CONTROL_PLANE_ONLY" == 1 ]] && wait_http "http://${h}:${p}/health" 5; then log_warn "API is up but not ready (expected without Triton in control-plane-only mode)" else - fails+=("API /health not ready: ./openprocessor logs yolo-api") + fails+=("API /health not ready: ./openprocessor logs api") fi elif [[ "$OP_CONTROL_PLANE_ONLY" != 1 ]]; then local cur=0 vlm_on=0 _has_tier curation && cur=1 { _has_tier vlm || [[ -n "${OP_VLM_URL:-}" ]]; } && vlm_on=1 - if ! dc exec -T yolo-api python -c "$_API_PROBE_PY" "$cur" "$vlm_on"; then - fails+=("API functional probes failed: ./openprocessor logs yolo-api") + if ! dc exec -T api python -c "$_API_PROBE_PY" "$cur" "$vlm_on"; then + fails+=("API functional probes failed: ./openprocessor logs api") fi fi if _has_tier segmenter; then @@ -2786,13 +2786,13 @@ do_install() { done SELECTED_TIERS="core" if [[ ",${OP_TIERS}," == *",cropwright,"* ]]; then SELECTED_TIERS="core cropwright"; fi - # yolo-api reserves a GPU in the base compose; without one it could + # api reserves a GPU in the base compose; without one it could # never start, so the control-plane install drops that reservation. local cver cver="$(dc version --short 2>/dev/null || true)" _version_ge "${cver#v}" "$COMPOSE_MIN_OVERRIDE" \ || die "Docker Compose ${COMPOSE_MIN_OVERRIDE}+ is needed for --control-plane-only (found '${cver:-unknown}')" - ( umask 022; printf 'services:\n yolo-api:\n deploy: !reset {}\n' > "${OP_DIR}/docker-compose.cpu.yml" ) + ( umask 022; printf 'services:\n api:\n deploy: !reset {}\n' > "${OP_DIR}/docker-compose.cpu.yml" ) GPU_PLAN_SUMMARY="none (control-plane-only)" else if [[ -n "$OP_TIERS" ]]; then @@ -2994,8 +2994,8 @@ do_install() { MODEL_SETUP_LOGDIR="${OP_DIR}/.install/logs" local group_rc=0 if [[ "$OP_CONTROL_PLANE_ONLY" == 1 ]]; then - dc up -d opensearch || die "compose up opensearch failed" - dc up -d --no-deps yolo-api || die "compose up yolo-api failed" + dc up -d --remove-orphans opensearch || die "compose up opensearch failed" + dc up -d --no-deps api || die "compose up api failed" elif [[ "$OP_SKIP_MODELS" == 1 ]]; then log_info "--skip-models: engines are not exported; run ./openprocessor models install later" else diff --git a/src/README.md b/src/README.md index db0b42170..96bc0f314 100644 --- a/src/README.md +++ b/src/README.md @@ -420,7 +420,7 @@ Settings managed via `config/settings.py` with environment variables: docker compose up -d # View logs -docker compose logs -f yolo-api +docker compose logs -f api # Stop services docker compose down diff --git a/src/routers/curation/bakeoff.py b/src/routers/curation/bakeoff.py index 8fcf684c2..d27311c0a 100644 --- a/src/routers/curation/bakeoff.py +++ b/src/routers/curation/bakeoff.py @@ -1,6 +1,6 @@ """Model comparison (bake-off) API: enqueue comparison jobs and serve their results. -yolo-api pins an older Ultralytics release and cannot load every model +api pins an older Ultralytics release and cannot load every model family the harness scores, so it does NOT run the comparison itself. It resolves the request (eval datasets, training runs, class mappings; see ``src/services/curation/bakeoff_jobs.py``) into a ``.job.json`` in diff --git a/src/routers/curation/pipeline.py b/src/routers/curation/pipeline.py index 05ed46101..a01bb4266 100644 --- a/src/routers/curation/pipeline.py +++ b/src/routers/curation/pipeline.py @@ -162,7 +162,7 @@ async def _run_auto_label( # R6-1a fix (Blocker, W3/W4 round-6 review): auto_label_worker runs # this in its OWN container/process, with its own config-store # snapshot that starts empty and was refreshed nowhere on the job - # path -- `/start`'s own `ensure_fresh` runs in the yolo-api + # path -- `/start`'s own `ensure_fresh` runs in the api # process, and a different process's in-memory snapshot does not # inherit that. Without this, a `(name, revision)` pin resolved at # `/start` time 404s here every time ("unknown prompt pack") the diff --git a/src/routers/health.py b/src/routers/health.py index 6eb676801..2368eede7 100644 --- a/src/routers/health.py +++ b/src/routers/health.py @@ -63,8 +63,8 @@ async def _probe_triton() -> tuple[bool, str]: valid readiness signal: it stays 0 until the first real infer call lands on the pool. Combined with ``depends_on: service_healthy`` on dependent containers, that produces a startup deadlock — dependents - wait for yolo-api to report healthy, but nothing is calling Triton - through yolo-api to wake the pool, so the count never moves. + wait for api to report healthy, but nothing is calling Triton + through api to wake the pool, so the count never moves. ``is_server_live()`` actively probes Triton via the async client (created lazily on first call). The side effect is exactly what we diff --git a/src/services/config_store/store.py b/src/services/config_store/store.py index 69cffef95..26a5e5ed5 100644 --- a/src/services/config_store/store.py +++ b/src/services/config_store/store.py @@ -581,7 +581,7 @@ async def startup_bootstrap_config_store_safe() -> Any: a broad ``except`` that returned ``None`` on any failure there -- including a plain unreachable-OpenSearch startup or a lost index-create race, both realistic at cold-stack-start (production - ``yolo-api`` has no ``opensearch: service_healthy`` gate and runs 32 + ``api`` has no ``opensearch: service_healthy`` gate and runs 32 workers). Nothing then ever retried, so that worker had no poll task for its entire lifetime (M4 inert), contrary to what this docstring used to claim. Fixed by mirroring diff --git a/src/services/curation/autolabel/job.py b/src/services/curation/autolabel/job.py index e1dd9a21e..86959c443 100644 --- a/src/services/curation/autolabel/job.py +++ b/src/services/curation/autolabel/job.py @@ -2,7 +2,7 @@ The actual pipeline now runs in a **dedicated long-lived worker container** (``curation-auto-label-worker``, entrypoint -``scripts/curation/auto_label_worker.py``). yolo-api just drops a +``scripts/curation/auto_label_worker.py``). api just drops a trigger JSON file on the shared ``/jobs`` volume and returns 202. The worker picks it up, runs the pipeline, and writes back to the same state.json the SSE endpoint already tails. @@ -193,9 +193,9 @@ def reconcile_orphaned_jobs() -> bool: Unlike the other three job modules in this package, the auto-label pipeline runs in a *separate* long-lived worker container (``curation-auto-label-worker``) with its own restart lifecycle — - restarting yolo-api does not kill that worker, so most of the time + restarting api does not kill that worker, so most of the time there is nothing to reconcile here. This still matters for the case - where yolo-api itself was down (and so never got to run this check) + where api itself was down (and so never got to run this check) while the worker died mid-run: without this, nothing repairs ``state.json`` until the next status poll. A pending, not-yet-claimed trigger is left alone — the worker container may simply not have diff --git a/src/services/curation/clustering/methods/ivf_store.py b/src/services/curation/clustering/methods/ivf_store.py index 71da428ae..d5e65d983 100644 --- a/src/services/curation/clustering/methods/ivf_store.py +++ b/src/services/curation/clustering/methods/ivf_store.py @@ -6,7 +6,7 @@ * the **auto-label worker** trains them (periodic recluster) and writes them here; -* the **ingest path** (yolo-api) loads them once and assigns every new +* the **ingest path** (api) loads them once and assigns every new residual crop to its nearest centroid at ingest time — no batch wait. Both processes resolve the store directory from the currently bound diff --git a/src/services/curation/clustering/refine.py b/src/services/curation/clustering/refine.py index 1af44f35c..11e5d92e1 100644 --- a/src/services/curation/clustering/refine.py +++ b/src/services/curation/clustering/refine.py @@ -50,7 +50,7 @@ # Refinement thresholds. # > this — refinement is skipped. The refine path runs sklearn AHC with NO # connectivity graph, so it builds a FULL pairwise distance matrix: ~8*n^2 -# bytes (float64). Rough transient RAM in the yolo-api process: +# bytes (float64). Rough transient RAM in the api process: # 2000 -> ~32 MB 5000 -> ~200 MB 8000 -> ~512 MB 12000 -> ~1.15 GB # Plus ~4 KB/member to fetch embeddings. It's fast (AHC fit is seconds even # at 8k) and off-loaded to a worker thread, so the binding constraint is RAM, @@ -309,7 +309,7 @@ async def refine_members( ) # Off-load to a worker thread so a large cluster (close to # MAX_REFINE_MEMBERS) doesn't block the FastAPI event loop -- refine runs - # in the yolo-api process, so a sync fit here would starve every other + # in the api process, so a sync fit here would starve every other # request. sub_labels = await asyncio.to_thread(clusterer.fit_predict, embeddings) n_subclusters = int(sub_labels.max() + 1) if len(sub_labels) > 0 else 0 diff --git a/src/services/curation/dataset_import/runner.py b/src/services/curation/dataset_import/runner.py index f51c772c7..335d89ddc 100644 --- a/src/services/curation/dataset_import/runner.py +++ b/src/services/curation/dataset_import/runner.py @@ -1,6 +1,6 @@ """The import job: claim, run in chunks, resume, cancel (W10.11). -Runs in-process as an ``asyncio.Task`` in one ``yolo-api`` worker; every +Runs in-process as an ``asyncio.Task`` in one ``api`` worker; every fact another process needs lives on disk (:mod:`.store`), so status, cancel and resume work from whichever worker a request lands on. diff --git a/src/services/curation/dataset_import/store.py b/src/services/curation/dataset_import/store.py index 9da999135..38e03bdf5 100644 --- a/src/services/curation/dataset_import/store.py +++ b/src/services/curation/dataset_import/store.py @@ -2,7 +2,7 @@ One directory per import under the bound project's imports dir (``OP_DATASET_IMPORTS_DIR/projects///``, on the shared -jobs volume so every ``yolo-api`` worker process sees it): +jobs volume so every ``api`` worker process sees it): ``request.json`` (the request as accepted), ``mapping.json`` (the resolved mapping with real class ids, plus the pinned region profile: a resume in diff --git a/src/services/curation/event_hub.py b/src/services/curation/event_hub.py index be02ca23c..14393fdc5 100644 --- a/src/services/curation/event_hub.py +++ b/src/services/curation/event_hub.py @@ -9,7 +9,7 @@ Design (S-3, file-backed bus): -- ``yolo-api`` runs 8 uvicorn *worker processes*; an in-process-only hub +- ``api`` runs 8 uvicorn *worker processes*; an in-process-only hub (the original design) means an SSE client connected to worker 3 never sees an event published by worker 7. The bus fixes that with one shared append-only JSONL log per state dir diff --git a/src/services/curation/file_job.py b/src/services/curation/file_job.py index 3aa265f23..e4b7639d6 100644 --- a/src/services/curation/file_job.py +++ b/src/services/curation/file_job.py @@ -2,7 +2,7 @@ ``state.json`` (atomic temp+rename), ``heartbeat``, ``cancel.flag`` and a cross-process start lock under one directory, on the shared ``/jobs`` -volume: every ``yolo-api`` worker process sees the same files, so status, +volume: every ``api`` worker process sees the same files, so status, cancel and singleton start are correct whichever process a request lands on (the multi-worker fix ``probe_job`` documents). diff --git a/src/services/curation/item_scores/job.py b/src/services/curation/item_scores/job.py index 7bf5b6f3e..c4f2631f8 100644 --- a/src/services/curation/item_scores/job.py +++ b/src/services/curation/item_scores/job.py @@ -3,7 +3,7 @@ Mirrors :mod:`src.services.curation.auto_label_job`'s state.json / heartbeat / cancel.flag file-backed conventions, simplified for in-process execution — there is no separate scores worker container; the job runs as an -``asyncio`` background task inside the yolo-api process itself, started +``asyncio`` background task inside the api process itself, started by the router handler and polled via the same state-file pattern the labeler already knows how to render. @@ -164,7 +164,7 @@ def start_job(opensearch: AsyncOpenSearch, scorer_names: list[str]) -> dict[str, 'running' happens before any ``await`` point, so a second call issued immediately after (even from a concurrent request) sees the 'running' state without racing the background task's own progress. That - in-process ordering isn't enough on its own though — ``yolo-api`` runs + in-process ordering isn't enough on its own though — ``api`` runs under ``--workers=N``, so two calls landing on *different* worker processes at nearly the same instant could both pass ``_is_busy()`` before either has written ``'running'``. The diff --git a/src/services/curation/job_lock.py b/src/services/curation/job_lock.py index 344d14637..b1a33d238 100644 --- a/src/services/curation/job_lock.py +++ b/src/services/curation/job_lock.py @@ -4,7 +4,7 @@ Every file-backed job module here (``item_scores.job``, ``probe_job``, ...) guards "start a new run" with a read-state / write-``'running'``-state -pair. That pair is only atomic *within one process*: ``yolo-api`` runs +pair. That pair is only atomic *within one process*: ``api`` runs under ``--workers=N``, i.e. N separate OS processes, so two concurrent ``start_job()`` calls landing on different worker processes at nearly the same instant can both read ``status != 'running'`` before either has diff --git a/src/services/curation/probe_job.py b/src/services/curation/probe_job.py index 4ba8ef5ad..69c3b50d2 100644 --- a/src/services/curation/probe_job.py +++ b/src/services/curation/probe_job.py @@ -3,7 +3,7 @@ ``POST /curation/probe/run`` doesn't block the request on a potentially-long CPU/GPU inference pass over the whole items index. -Runs in-process (an ``asyncio.Task`` inside the yolo-api process itself, +Runs in-process (an ``asyncio.Task`` inside the api process itself, sharing the same GPU-claim path ``POST /train/start`` uses) -- unlike :mod:`src.services.curation.autolabel.job`, there is no separate probe worker container, so this mirrors :mod:`src.services.curation.item_scores.job` @@ -11,7 +11,7 @@ state.json/heartbeat/cancel.flag convention instead of ``autolabel.job``'s trigger-file dispatch to a long-lived worker. -**Multi-worker correctness (2026-09-25 fix).** ``yolo-api`` runs under +**Multi-worker correctness (2026-09-25 fix).** ``api`` runs under ``uvicorn --workers=8`` -- eight separate OS processes. The previous module-level ``_ProbeJobState`` dataclass plus ``_task`` handle were invisible across workers: ``POST /probe/run`` landing on worker A and a @@ -24,7 +24,7 @@ by moving to the same on-disk, atomically-written state.json + heartbeat + cancel.flag convention ``item_scores.job``/``embedding_viz`` already use, resolved fresh per call from ``OP_PROBE_JOBS_DIR`` (default -``/jobs/probe`` -- the shared ``/jobs`` volume every yolo-api worker +``/jobs/probe`` -- the shared ``/jobs`` volume every api worker process mounts, consistent with ``item_scores``' ``/jobs/scores`` and ``embedding_viz``'s ``/jobs/viz``). diff --git a/src/services/curation/region_drain.py b/src/services/curation/region_drain.py index 9f01fa793..d70c57e9b 100644 --- a/src/services/curation/region_drain.py +++ b/src/services/curation/region_drain.py @@ -7,7 +7,7 @@ agrees, and so the parameters (``poll_interval_s`` / ``stable_polls``) are served, not hardcoded twice. -**Multi-worker correctness (2026-09-25 fix).** ``yolo-api`` runs under +**Multi-worker correctness (2026-09-25 fix).** ``api`` runs under ``--workers=N`` -- separate OS processes. This used to keep the streak counter (``_StreakState``) in module memory: an ingest walker polling this endpoint every ``poll_interval_s`` lands on essentially a random diff --git a/src/services/labeling/vlm_url_policy.py b/src/services/labeling/vlm_url_policy.py index bf1472581..fcf922a66 100644 --- a/src/services/labeling/vlm_url_policy.py +++ b/src/services/labeling/vlm_url_policy.py @@ -48,7 +48,7 @@ 'opensearch-dashboards', 'triton-server', 'triton-sdk', - 'yolo-api', + 'api', 'op-api', 'segmenter', 'curation-detection-worker', diff --git a/src/services/projects/combine/store.py b/src/services/projects/combine/store.py index 9aa56b4bd..a42fbd136 100644 --- a/src/services/projects/combine/store.py +++ b/src/services/projects/combine/store.py @@ -1,7 +1,7 @@ """The on-disk record of combine jobs. One directory per job under ``OP_COMBINE_JOBS_DIR`` (shared jobs volume, so -every ``yolo-api`` worker process sees it): the W10 import store's layout +every ``api`` worker process sees it): the W10 import store's layout (``request.json``, ``mapping.json``, ``state.json`` / ``heartbeat`` / ``cancel.flag``, the write-ahead ledger, ``chunks_done.jsonl``), reused as is. A combine is global (it acts on several projects), so its directory is not diff --git a/src/services/projects/delete.py b/src/services/projects/delete.py index 0e33d2faa..ba82ae142 100644 --- a/src/services/projects/delete.py +++ b/src/services/projects/delete.py @@ -57,7 +57,7 @@ # 'active'. # # P3F pass-4 F1 correction: this guard is PER-WORKER-PROCESS ONLY -- -# yolo-api runs `--workers=32` (docker-compose.yml), and each worker has +# api runs `--workers=32` (docker-compose.yml), and each worker has # its own, separate, empty copy of this set. It stops the race only # within the one worker process that happens to handle both the # original DELETE and its retry. A retried DELETE that a proxy/client diff --git a/src/services/training/preflight_checks.py b/src/services/training/preflight_checks.py index 596f40464..0238ea501 100644 --- a/src/services/training/preflight_checks.py +++ b/src/services/training/preflight_checks.py @@ -208,7 +208,7 @@ def _nearest_existing(path: str) -> Path: def resolve_disk_check_path(spec: TrainJobSpec) -> str: """Pick the path to stat for the free-disk check. - Previously hardcoded to a host data-volume root — inside the yolo-api + Previously hardcoded to a host data-volume root — inside the api container, only specific subpaths under that root (e.g. a deployment-specific training-data mount, see the deployment's own compose overlay) are bind-mounted from the real training-data volume; diff --git a/src/services/training/promote_json.py b/src/services/training/promote_json.py index 523722464..c9b9daf00 100644 --- a/src/services/training/promote_json.py +++ b/src/services/training/promote_json.py @@ -3,7 +3,7 @@ Two writers touch the file: ``PUT .../models/{name}/sharing`` (the owner's opt-in, with optimistic concurrency on ``sharing_revision``) and a -re-promote (a new version of the same model). ``yolo-api`` runs several +re-promote (a new version of the same model). ``api`` runs several worker processes, so every read-compare-write holds a per-model ``flock`` (:func:`~src.services.curation.job_lock.exclusive_file_lock`) and every write is a temp file plus rename, so a crash never leaves a torn diff --git a/src/services/training/triton_repo.py b/src/services/training/triton_repo.py index ca40697cf..0aa9ef0d4 100644 --- a/src/services/training/triton_repo.py +++ b/src/services/training/triton_repo.py @@ -30,7 +30,7 @@ def resolve_triton_models_dir() -> Path: return Path(override) if override else DEFAULT_TRITON_MODELS_DIR -# Triton's HTTP control endpoint. The yolo-api container shares the +# Triton's HTTP control endpoint. The api container shares the # triton_net network so this resolves through Docker DNS. DEFAULT_TRITON_HTTP_URL = 'http://triton-server:8000' diff --git a/src/ultralytics_patches/README.md b/src/ultralytics_patches/README.md index 6fe37698f..6a7822507 100644 --- a/src/ultralytics_patches/README.md +++ b/src/ultralytics_patches/README.md @@ -78,7 +78,7 @@ See `export/export_models.py` for complete export script. ```bash # Export all YOLOv11 models with end2end NMS -docker compose exec yolo-api python /app/export/export_models.py --formats trt_end2end +docker compose exec api python /app/export/export_models.py --formats trt_end2end ``` ## Output Format diff --git a/tests/curation/test_event_bus_cross_process.py b/tests/curation/test_event_bus_cross_process.py index 191a6ea4f..ca91f342e 100644 --- a/tests/curation/test_event_bus_cross_process.py +++ b/tests/curation/test_event_bus_cross_process.py @@ -188,17 +188,17 @@ def test_bulk_writer_event_api_url_falls_back_to_api_base_url( monkeypatch: pytest.MonkeyPatch, ) -> None: """OP_EVENT_API_URL is the explicit override; with only OP_API_BASE_URL - set (the trainer/campaign convention for "where yolo-api lives"), the + set (the trainer/campaign convention for "where api lives"), the detection worker's publish path should still resolve a URL instead of silently going dark.""" monkeypatch.delenv('OP_EVENT_API_URL', raising=False) - monkeypatch.setenv('OP_API_BASE_URL', 'http://yolo-api:8000/') + monkeypatch.setenv('OP_API_BASE_URL', 'http://api:8000/') monkeypatch.delenv('OP_API', raising=False) from scripts.curation.worker import bulk_writer importlib.reload(bulk_writer) - assert bulk_writer._EVENT_API_URL == 'http://yolo-api:8000' + assert bulk_writer._EVENT_API_URL == 'http://api:8000' @pytest.mark.usefixtures('_reload_bulk_writer_after_env_restored') diff --git a/tests/curation/test_probe_job.py b/tests/curation/test_probe_job.py index 0ee1c9dde..1f8a24a49 100644 --- a/tests/curation/test_probe_job.py +++ b/tests/curation/test_probe_job.py @@ -1,7 +1,7 @@ """The file-backed probe job wrapper (src.services.curation.probe_job). Multi-worker correctness is the point of this module (2026-09-25 fix): -``yolo-api`` runs under ``--workers=N`` -- separate OS processes that +``api`` runs under ``--workers=N`` -- separate OS processes that don't share Python objects. Every test here points ``OP_PROBE_JOBS_DIR`` at a fresh ``tmp_path`` and, wherever a test claims to simulate "a second process", it deliberately avoids relying on any module-level Python diff --git a/tests/curation/test_region_dependency_health.py b/tests/curation/test_region_dependency_health.py index cd553346a..169209caa 100644 --- a/tests/curation/test_region_dependency_health.py +++ b/tests/curation/test_region_dependency_health.py @@ -4,7 +4,7 @@ Persisted to a state.json under ``OP_REGION_DRAIN_STATE_DIR`` (same shared-``/jobs``-volume convention as ``region_drain.py``'s streak state), so the "unavailable since" timestamp is stable across -``yolo-api --workers`` processes and repeated polls, rather than +``api --workers`` processes and repeated polls, rather than resetting to "just now" every time. """ diff --git a/tests/curation/test_region_drain_verdict.py b/tests/curation/test_region_drain_verdict.py index 2299fd8b4..7f51bade2 100644 --- a/tests/curation/test_region_drain_verdict.py +++ b/tests/curation/test_region_drain_verdict.py @@ -2,7 +2,7 @@ Persisted to a state.json under ``OP_REGION_DRAIN_STATE_DIR`` rather than kept in module memory, so the verdict is correct regardless of which -``yolo-api --workers`` process answers a given poll -- every test here +``api --workers`` process answers a given poll -- every test here points that env var at a per-test ``tmp_path``. """ diff --git a/tests/curation/test_vlm_hot_switch_api.py b/tests/curation/test_vlm_hot_switch_api.py index 6a627dc03..8a788eb11 100644 --- a/tests/curation/test_vlm_hot_switch_api.py +++ b/tests/curation/test_vlm_hot_switch_api.py @@ -273,7 +273,7 @@ def test_the_deny_policy_refuses_external_even_with_the_flag( ('http://2852039166/v1', 'vlm_url_denied_address'), ('http://[::ffff:169.254.169.254]/v1', 'vlm_url_denied_address'), ('http://opensearch:9200/v1', 'vlm_url_denied_internal_service'), - ('http://yolo-api:4603/v1', 'vlm_url_denied_internal_service'), + ('http://api:4603/v1', 'vlm_url_denied_internal_service'), ], ) def test_forbidden_targets_cannot_be_created_validated_or_probed( diff --git a/tests/installer/shims/docker b/tests/installer/shims/docker index 59acdc9ab..cb6682bd4 100755 --- a/tests/installer/shims/docker +++ b/tests/installer/shims/docker @@ -191,7 +191,7 @@ ${cservice} "triton-server": { "container_name": "${cpn}-triton" }, - "yolo-api": { + "api": { "container_name": "${cpn}-api" } }, diff --git a/tests/installer/test_cli_vlm.py b/tests/installer/test_cli_vlm.py index b134e8bec..71b9b3168 100644 --- a/tests/installer/test_cli_vlm.py +++ b/tests/installer/test_cli_vlm.py @@ -120,23 +120,21 @@ def test_use_rewrites_env_recreates_the_vlm_probes_and_unpauses( def test_the_first_use_migrates_to_the_stable_alias_and_recreates_the_running_services( shimmed: Shimmed, stack: Path ) -> None: - (shimmed.state / 'running_services').write_text( - 'yolo-api\ncuration-detection-worker\nsegmenter\n' - ) + (shimmed.state / 'running_services').write_text('api\ncuration-detection-worker\nsegmenter\n') result = use(shimmed, stack) assert result.returncode == 0, result.stdout + result.stderr assert 'this one time the API and workers are recreated too' in result.stdout + result.stderr assert env_value(stack, 'VLM_SERVED_MODEL_NAME') == 'local-vlm' assert env_value(stack, 'OP_VLM_MODEL') == 'local-vlm' ups = [c for c in compose_calls(shimmed) if 'up -d' in c] - assert ups == ['up -d vlm yolo-api curation-detection-worker'] # never the segmenter + assert ups == ['up -d vlm api curation-detection-worker'] # never the segmenter def test_once_on_the_alias_only_the_vlm_container_is_recreated( shimmed: Shimmed, stack: Path ) -> None: set_env(stack, VLM_SERVED_MODEL_NAME='local-vlm', OP_VLM_MODEL='local-vlm') - (shimmed.state / 'running_services').write_text('yolo-api\ncurie\n') + (shimmed.state / 'running_services').write_text('api\ncurie\n') assert use(shimmed, stack).returncode == 0 assert [c for c in compose_calls(shimmed) if 'up -d' in c] == ['up -d vlm'] @@ -253,7 +251,7 @@ def test_an_unreachable_api_container_aborts_before_touching_env( before = (stack / '.env').read_text() result = use(shimmed, stack) assert result.returncode == 1 - assert 'cannot reach the yolo-api container' in result.stdout + result.stderr + assert 'cannot reach the api container' in result.stdout + result.stderr assert (stack / '.env').read_text() == before @@ -416,11 +414,11 @@ def test_a_changed_image_cap_recreates_the_running_clients(shimmed: Shimmed, sta OP_VLM_MODEL='local-vlm', OP_VLM_MAX_IMAGES_PER_CALL='4', ) - (shimmed.state / 'running_services').write_text('yolo-api\ncuration-vlm-worker\nsegmenter\n') + (shimmed.state / 'running_services').write_text('api\ncuration-vlm-worker\nsegmenter\n') assert use(shimmed, stack).returncode == 0 assert env_value(stack, 'OP_VLM_MAX_IMAGES_PER_CALL') == '8' assert [c for c in compose_calls(shimmed) if 'up -d' in c] == [ - 'up -d vlm yolo-api curation-vlm-worker' + 'up -d vlm api curation-vlm-worker' ] diff --git a/tests/installer/test_lifecycle.py b/tests/installer/test_lifecycle.py index 2822828f3..79680d2c0 100644 --- a/tests/installer/test_lifecycle.py +++ b/tests/installer/test_lifecycle.py @@ -85,7 +85,7 @@ def test_install_performs_every_step_in_order(shimmed: Shimmed) -> None: preflight = index_of(m, 'python -m export.preflight') yolo = index_of(m, 'export/export_models.py') up = index_of(m, ' up -d --remove-orphans') - probes = index_of(m, ' exec -T yolo-api python -c') + probes = index_of(m, ' exec -T api python -c') assert pull < probe < triton < preflight < yolo < up < probes for line in m: if 'docker compose' in line: diff --git a/tests/installer/test_setup_dry_run.py b/tests/installer/test_setup_dry_run.py index 438b1c9e7..22309840d 100644 --- a/tests/installer/test_setup_dry_run.py +++ b/tests/installer/test_setup_dry_run.py @@ -103,8 +103,8 @@ def test_dry_run_makes_no_mutating_docker_call_and_plans_every_step(shimmed: Shi 'docker pull davidamacey/openprocessor-triton@sha256:', 'docker run --rm --gpus device=0', 'up -d triton-server', - 'yolo-api python -m export.preflight', - '--entrypoint cp yolo-api -rn /opt/openprocessor/model_repo_seed/. /app/models/', + 'api python -m export.preflight', + '--entrypoint cp api -rn /opt/openprocessor/model_repo_seed/. /app/models/', 'export/export_models.py', 'export_paddleocr_rec.py', 'up -d --remove-orphans', @@ -200,7 +200,7 @@ def test_unprefixed_container_names_exit_3(shimmed: Shimmed) -> None: shimmed.flag( 'config.json', '{\n "networks": {\n "triton_net": {\n "name": "opinst-test_triton_net"\n }\n },\n' - ' "services": {\n "yolo-api": {\n "container_name": "openprocessor-api"\n }\n }\n}\n', + ' "services": {\n "api": {\n "container_name": "openprocessor-api"\n }\n }\n}\n', ) result = dry(shimmed) assert result.returncode == 3 @@ -211,7 +211,7 @@ def test_wrong_network_name_exits_3(shimmed: Shimmed) -> None: shimmed.flag( 'config.json', '{\n "networks": {\n "triton_net": {\n "name": "openprocessor_triton_net"\n }\n },\n' - ' "services": {\n "yolo-api": {\n "container_name": "opinst-test-api"\n }\n }\n}\n', + ' "services": {\n "api": {\n "container_name": "opinst-test-api"\n }\n }\n}\n', ) assert dry(shimmed).returncode == 3 @@ -483,8 +483,8 @@ def test_control_plane_only_starts_api_and_opensearch_only(shimmed: Shimmed) -> result = dry(shimmed, '--cpu', '--control-plane-only') assert result.returncode == 0, result.stderr dry_compose = [ln for ln in result.stdout.splitlines() if ln.startswith('DRY: docker compose')] - assert any(ln.endswith(' up -d opensearch') for ln in dry_compose) - assert any(ln.endswith(' up -d --no-deps yolo-api') for ln in dry_compose) + assert any(ln.endswith(' up -d --remove-orphans opensearch') for ln in dry_compose) + assert any(ln.endswith(' up -d --no-deps api') for ln in dry_compose) assert not any('triton-server' in ln for ln in dry_compose) assert all('docker-compose.cpu.yml' in ln for ln in dry_compose) assert configure(shimmed, '--cpu', '--control-plane-only').returncode == 0 diff --git a/tests/integration/test_prometheus_scrape_config.py b/tests/integration/test_prometheus_scrape_config.py index 6dcc4f805..cc2237c50 100644 --- a/tests/integration/test_prometheus_scrape_config.py +++ b/tests/integration/test_prometheus_scrape_config.py @@ -37,7 +37,7 @@ def _jobs(prometheus_config: dict) -> dict[str, dict]: def test_expected_scrape_jobs_present(prometheus_config: dict) -> None: jobs = _jobs(prometheus_config) - for expected in ('triton', 'yolo-api', 'node', 'dcgm', 'loki'): + for expected in ('triton', 'api', 'node', 'dcgm', 'loki'): assert expected in jobs, f'scrape job {expected!r} missing from prometheus.yml' diff --git a/tests/projects/test_ma1_status_write_races.py b/tests/projects/test_ma1_status_write_races.py index e34f18a87..6ddd9b208 100644 --- a/tests/projects/test_ma1_status_write_races.py +++ b/tests/projects/test_ma1_status_write_races.py @@ -169,7 +169,7 @@ def test_second_finish_on_a_different_worker_cannot_destroy_a_project_a_first_fi ) -> None: """P3F pass-4 F1 -- the P3 review's pass-3 confirmation found the prior pass's ``test_second_finish_cannot_delete_a_project_...`` test - (above) proves nothing about the ACTUAL production topology: yolo-api + (above) proves nothing about the ACTUAL production topology: api runs ``--workers=32``, and ``delete._FINISH_IN_PROGRESS`` is a per-worker-process set. A re-DELETE that lands on a DIFFERENT worker (31 times out of 32) has its own, separate, empty copy of that guard diff --git a/tests/test_bind_mount_placeholder_dirs.py b/tests/test_bind_mount_placeholder_dirs.py index 13d120dc3..189a0ae35 100644 --- a/tests/test_bind_mount_placeholder_dirs.py +++ b/tests/test_bind_mount_placeholder_dirs.py @@ -2,7 +2,7 @@ must exist, host-user-owned, before compose ever runs -- without breaking `git pull`. -``yolo-api`` bind-mounts ``./test_images``. On a fresh clone that host +``api`` bind-mounts ``./test_images``. On a fresh clone that host directory doesn't exist, so Docker auto-creates it **root-owned** on the very first ``docker compose up``, and ``make download-test-images`` (running as the host user) then fails to write into it. diff --git a/tests/test_compose_contract.py b/tests/test_compose_contract.py index 9e0d6979b..d990d367e 100644 --- a/tests/test_compose_contract.py +++ b/tests/test_compose_contract.py @@ -228,7 +228,7 @@ def test_no_duplicate_host_ports() -> None: # than a dev checkout's live `git rev-parse HEAD` fallback. # ============================================================================= -_BUILD_SHA_SERVICES = ('yolo-api', 'curation-trainer', 'curation-evaluator') +_BUILD_SHA_SERVICES = ('api', 'curation-trainer', 'curation-evaluator') _BUILD_SHA_DOCKERFILES = ( REPO_ROOT / 'Dockerfile', @@ -270,7 +270,7 @@ def test_evaluator_sees_exports_at_the_api_path() -> None: """Bake-off eval datasets are exports: the evaluator must read them at the path the API resolved (``./data/projects//exports`` -> ``/app/data``).""" services = _services() - api_mounts = services['yolo-api'].get('volumes') or [] + api_mounts = services['api'].get('volumes') or [] evaluator_mounts = services['curation-evaluator'].get('volumes') or [] assert any(str(v).startswith('./data:/app/data') for v in api_mounts) assert './data:/app/data:ro' in evaluator_mounts @@ -295,20 +295,20 @@ def _value(env: list[str], key: str) -> str | None: evaluator_url = _value(evaluator_env, 'MLFLOW_TRACKING_URI') assert trainer_url is not None assert evaluator_url == trainer_url - api_env = services['yolo-api'].get('environment') or [] + api_env = services['api'].get('environment') or [] assert _value(api_env, 'MLFLOW_TRACKING_URI') == trainer_url, ( 'project delete cleans the MLflow experiment over REST from the API' ) def test_api_evaluator_segmenter_gpu_ids_are_env_driven_not_hardcoded() -> None: - """F-2 (fresh-start E2E findings 2026-09-25): yolo-api, curation-evaluator + """F-2 (fresh-start E2E findings 2026-09-25): api, curation-evaluator and segmenter used to hardcode device_ids: ['0'] regardless of TRITON_GPU_ID/VLM_GPU_ID, so a host whose free GPU wasn't 0 needed a compose edit to run them at all.""" services = _services() expectations = { - 'yolo-api': 'API_GPU_ID', + 'api': 'API_GPU_ID', 'curation-evaluator': 'EVALUATOR_GPU_ID', 'segmenter': 'SEGMENTER_GPU_ID', } @@ -333,13 +333,13 @@ def test_auto_label_worker_caps_blas_threads() -> None: def test_api_and_detection_worker_share_crop_cache() -> None: - """ST-1: yolo-api and curation-detection-worker must mount the SAME + """ST-1: api and curation-detection-worker must mount the SAME named crop-cache volume at the SAME target, with OP_CROP_CACHE_DIR set to that target in both -- otherwise the worker's cache reads never see what the API's ingest path wrote.""" services = _services() target = '/var/cache/openprocessor/crops' - for name in ('yolo-api', 'curation-detection-worker'): + for name in ('api', 'curation-detection-worker'): mounts = [str(v) for v in (services[name].get('volumes') or [])] matching = [m for m in mounts if m.endswith(f':{target}')] assert matching, f'{name} has no crop-cache mount at {target}: {mounts}' @@ -392,8 +392,8 @@ def test_curation_worker_healthchecks_use_liveness() -> None: assert not bad, 'expected worker_liveness-based healthchecks:\n' + '\n'.join(bad) -# Workers whose depends_on already names yolo-api directly (main compose -# doesn't have every curation worker depend on yolo-api — e.g. +# Workers whose depends_on already names api directly (main compose +# doesn't have every curation worker depend on api — e.g. # curation-detection-worker depends on triton-server/opensearch, and # curation-auto-label-worker only on opensearch, both by design, since # they're triggered via files, not a direct HTTP call at startup). @@ -401,26 +401,26 @@ def test_curation_worker_healthchecks_use_liveness() -> None: def test_curation_workers_depend_on_healthy_api() -> None: - """S-7: workers that depend on `yolo-api` wait for it to report healthy, + """S-7: workers that depend on `api` wait for it to report healthy, not merely started.""" services = _services() bad: list[str] = [] for name in _WORKERS_DEPENDING_ON_API: depends_on = services[name].get('depends_on') - if not isinstance(depends_on, dict) or 'yolo-api' not in depends_on: + if not isinstance(depends_on, dict) or 'api' not in depends_on: bad.append(name) continue - condition = (depends_on.get('yolo-api') or {}).get('condition') + condition = (depends_on.get('api') or {}).get('condition') if condition != 'service_healthy': bad.append(name) - assert not bad, f'expected depends_on.yolo-api.condition == service_healthy: {bad}' + assert not bad, f'expected depends_on.api.condition == service_healthy: {bad}' -def test_yolo_api_waits_for_a_healthy_opensearch() -> None: +def test_api_waits_for_a_healthy_opensearch() -> None: """Found live: the API boots, fails to create the core kNN indexes because OpenSearch is not up yet, and never retries them -- so the first /ingest creates them with dynamic mappings.""" - depends_on = _services()['yolo-api'].get('depends_on') + depends_on = _services()['api'].get('depends_on') assert isinstance(depends_on, dict), depends_on assert (depends_on.get('opensearch') or {}).get('condition') == 'service_healthy' @@ -438,10 +438,10 @@ def test_triton_accepts_the_clients_idle_keepalive_pings() -> None: assert interval <= 30_000 -def test_yolo_api_has_a_healthcheck() -> None: - """A `service_healthy` dependency on yolo-api is meaningless without one.""" +def test_api_has_a_healthcheck() -> None: + """A `service_healthy` dependency on api is meaningless without one.""" services = _services() - assert services['yolo-api'].get('healthcheck'), 'yolo-api needs a healthcheck' + assert services['api'].get('healthcheck'), 'api needs a healthcheck' def test_curation_mlflow_has_a_healthcheck() -> None: @@ -528,7 +528,7 @@ def test_no_hardcoded_published_host_ports() -> None: # override var docker-compose.yml pins it through. _CUSTOM_IMAGE_OVERRIDE_VARS = { 'triton-server': 'OP_TRITON_IMAGE', - 'yolo-api': 'OP_API_IMAGE', + 'api': 'OP_API_IMAGE', 'curation-detection-worker': 'OP_API_IMAGE', 'curation-vlm-worker': 'OP_API_IMAGE', 'curation-auto-label-worker': 'OP_API_IMAGE', @@ -574,10 +574,10 @@ def test_dev_overlay_carries_every_build_block_and_source_mount() -> None: for name in _CUSTOM_IMAGE_OVERRIDE_VARS: assert name in dev_services, f'{name} missing from docker-compose.dev.yml' assert 'build' in dev_services[name], f'{name} missing build: in docker-compose.dev.yml' - yolo_api_mounts = [str(v) for v in dev_services['yolo-api'].get('volumes') or []] + api_mounts = [str(v) for v in dev_services['api'].get('volumes') or []] for prefix in _FORBIDDEN_BASE_MOUNT_PREFIXES: - assert any(m.startswith(prefix + ':') for m in yolo_api_mounts), ( - f'docker-compose.dev.yml/yolo-api missing a mount for {prefix}' + assert any(m.startswith(prefix + ':') for m in api_mounts), ( + f'docker-compose.dev.yml/api missing a mount for {prefix}' ) @@ -636,13 +636,13 @@ def test_no_latest_in_any_custom_image_fallback() -> None: def test_source_root_mounted_at_the_same_path_on_api_and_detection_worker() -> None: - """G-06: ingest resolves item paths against OP_SOURCE_ROOT on yolo-api; + """G-06: ingest resolves item paths against OP_SOURCE_ROOT on api; the detection worker re-reads the same items later. Without the same bind on both, every worker read fails with detection_failed/reason=image_unavailable.""" services = _services() target = '/data/source' - for name in ('yolo-api', 'curation-detection-worker', 'curation-auto-label-worker'): + for name in ('api', 'curation-detection-worker', 'curation-auto-label-worker'): mounts = [str(v) for v in (services[name].get('volumes') or [])] matching = [m for m in mounts if m.endswith((f':{target}:ro', f':{target}'))] assert matching, f'{name} has no source-root mount at {target}: {mounts}' @@ -673,7 +673,7 @@ def test_examples_reachable_on_api_and_detection_worker() -> None: dev_compose = _load_yaml(REPO_ROOT / 'docker-compose.dev.yml') dev_services = dev_compose['services'] target = '/app/examples' - for name in ('yolo-api', 'curation-detection-worker', 'curation-auto-label-worker'): + for name in ('api', 'curation-detection-worker', 'curation-auto-label-worker'): mounts = [str(v) for v in (dev_services.get(name, {}).get('volumes') or [])] assert any(m.endswith(f':{target}:ro') for m in mounts), ( f'{name} has no ./examples mount at {target} in docker-compose.dev.yml: {mounts}' @@ -691,7 +691,7 @@ def test_examples_reachable_on_api_and_detection_worker() -> None: # needs the class registry itself, so ./data is its own, narrower # requirement -- see test_class_registry_data_mounted_where_needed below. _IN_PROCESS_PIPELINE_SERVICES = ( - 'yolo-api', + 'api', 'curation-detection-worker', 'curation-auto-label-worker', ) @@ -704,12 +704,12 @@ def test_class_registry_data_mounted_where_needed() -> None: Without ./data mounted, the registry loads empty and any class-aware stage (the VLM auto-label stage, training, promote, eval) raises 'class_names or class_catalog must be supplied' or reads no classes - instead of running. yolo-api and curation-auto-label-worker read the + instead of running. api and curation-auto-label-worker read the registry directly; curation-detection-worker deliberately does not (it only ever writes unlabeled `*_proposal` detections).""" services = _services() target = '/app/data' - for name in ('yolo-api', 'curation-auto-label-worker'): + for name in ('api', 'curation-auto-label-worker'): mounts = [str(v) for v in (services[name].get('volumes') or [])] assert any(m.endswith((f':{target}', f':{target}:ro')) for m in mounts), ( f'{name} has no ./data mount at {target}: {mounts}' @@ -719,7 +719,7 @@ def test_class_registry_data_mounted_where_needed() -> None: def test_http_client_only_curation_services_stay_lightweight() -> None: """Documents *why* curation-vlm-worker/curation-cluster-refresh don't need the source-root/examples/data mounts above: they only ever talk - to yolo-api over HTTP (verified via each entrypoint's own imports, not + to api over HTTP (verified via each entrypoint's own imports, not inferred from mounts), so adding a heavier mount surface there would be unnecessary attack/complexity surface, not a missing-mount bug.""" services = _services() @@ -752,27 +752,27 @@ def test_vlm_service_is_opt_in_with_a_pinned_image() -> None: assert ':latest' not in image, f'vlm service image must be pinned, not :latest: {image!r}' -def test_yolo_api_carries_op_api_network_alias() -> None: +def test_api_carries_op_api_network_alias() -> None: """G-28: Cropwright's default API_UPSTREAM is http://op-api:8000; this alias lets that default resolve without every deployer overriding it.""" - networks = _services()['yolo-api'].get('networks') - assert isinstance(networks, dict), 'expected yolo-api networks: to carry aliases (dict form)' + networks = _services()['api'].get('networks') + assert isinstance(networks, dict), 'expected api networks: to carry aliases (dict form)' aliases = (networks.get('triton_net') or {}).get('aliases') or [] - assert 'op-api' in aliases, f'expected op-api in yolo-api triton_net aliases: {aliases}' + assert 'op-api' in aliases, f'expected op-api in api triton_net aliases: {aliases}' def test_gpu_arbiter_overlay_exists_and_mounts_docker_socket() -> None: """G-15: the docker socket must be opt-in (a separate overlay file), - never a default mount on yolo-api.""" + never a default mount on api.""" overlay_path = REPO_ROOT / 'docker-compose.gpu-arbiter.yml' assert overlay_path.is_file(), 'expected docker-compose.gpu-arbiter.yml overlay' with overlay_path.open() as fh: overlay = yaml.safe_load(fh) - mounts = [str(v) for v in (overlay['services']['yolo-api'].get('volumes') or [])] + mounts = [str(v) for v in (overlay['services']['api'].get('volumes') or [])] assert any('/var/run/docker.sock' in m for m in mounts), mounts # And the base compose must NOT already mount it (defeats the point of # an opt-in overlay). - base_mounts = [str(v) for v in (_services()['yolo-api'].get('volumes') or [])] + base_mounts = [str(v) for v in (_services()['api'].get('volumes') or [])] assert not any('docker.sock' in m for m in base_mounts), base_mounts @@ -790,18 +790,18 @@ def test_vlm_image_pinned_by_digest() -> None: assert '@sha256:' in image, image -def test_yolo_api_default_trainer_container_matches_the_trainer_service_name() -> None: +def test_api_default_trainer_container_matches_the_trainer_service_name() -> None: """F-72: /train/preflight's trainer-reachability probe (GpuArbiterConfig.trainer_container, env OP_GPU_ARBITER_TRAINER_CONTAINER) used to default to unset -- reporting the misleading "no trainer container configured" even while curation-trainer was up and healthy, because nothing ever wired the env var to the trainer service's own - container_name. yolo-api's environment must default + container_name. api's environment must default OP_GPU_ARBITER_TRAINER_CONTAINER to that exact value (still overridable), so the probe finds it with no extra config on any deployment running the `training` profile.""" services = _services() - api_env = services['yolo-api']['environment'] + api_env = services['api']['environment'] trainer_container_name = str(services['curation-trainer']['container_name']) env_map = {} @@ -810,7 +810,7 @@ def test_yolo_api_default_trainer_container_matches_the_trainer_service_name() - env_map[key] = value assert 'OP_GPU_ARBITER_TRAINER_CONTAINER' in env_map, ( - 'yolo-api must set a default OP_GPU_ARBITER_TRAINER_CONTAINER' + 'api must set a default OP_GPU_ARBITER_TRAINER_CONTAINER' ) default_expr = env_map['OP_GPU_ARBITER_TRAINER_CONTAINER'] # The env var's own default value (inside the outer ${VAR:-...}) must @@ -1054,11 +1054,11 @@ def test_the_catalog_default_matches_the_compose_default() -> None: def test_the_api_and_workers_get_the_local_endpoint_and_the_read_only_secrets_mount() -> None: services = yaml.safe_load(COMPOSE.read_text())['services'] - for name in ('yolo-api', 'curation-detection-worker'): + for name in ('api', 'curation-detection-worker'): env = ' '.join(services[name]['environment']) assert 'OP_LOCAL_VLM_ENDPOINT=${OP_LOCAL_VLM_ENDPOINT:-}' in env, name assert 'OP_LOCAL_VLM_GPU_TOTAL_MIB=${VLM_GPU_TOTAL_MIB:-}' in env, name - for name in ('yolo-api', 'curation-detection-worker', 'curation-auto-label-worker'): + for name in ('api', 'curation-detection-worker', 'curation-auto-label-worker'): assert './secrets/vlm:/run/secrets/op_vlm:ro' in services[name]['volumes'], name @@ -1151,7 +1151,7 @@ def dry_run(target: str) -> str: def test_api_service_aggregates_metrics_across_workers() -> None: """uvicorn --workers N keeps one registry per process; without a shared multiprocess dir a /metrics scrape reflects a single worker.""" - spec = _services()['yolo-api'] + spec = _services()['api'] env = spec['environment'] entries = dict(e.split('=', 1) for e in env) if isinstance(env, list) else env mp_dir = entries['PROMETHEUS_MULTIPROC_DIR'] diff --git a/tests/test_compose_gateway.py b/tests/test_compose_gateway.py index be667fde7..0dbe70eee 100644 --- a/tests/test_compose_gateway.py +++ b/tests/test_compose_gateway.py @@ -80,13 +80,13 @@ def _wrap( def test_gateway_switch_defaults_off_everywhere(tmp_path: Path) -> None: services = _services(tmp_path, {}) - for name in (*UI_SERVICES, 'cropwright', 'yolo-api'): + for name in (*UI_SERVICES, 'cropwright', 'api'): assert services[name]['environment']['OP_GATEWAY_SUBPATHS'] == 'false', name def test_gateway_switch_reaches_the_api_the_gateway_and_each_ui(tmp_path: Path) -> None: services = _services(tmp_path, {'OP_GATEWAY_SUBPATHS': 'true'}) - for name in (*UI_SERVICES, 'cropwright', 'yolo-api'): + for name in (*UI_SERVICES, 'cropwright', 'api'): assert services[name]['environment']['OP_GATEWAY_SUBPATHS'] == 'true', name diff --git a/tests/test_export_models_summary.py b/tests/test_export_models_summary.py index 17e6f0274..e7ef094fe 100644 --- a/tests/test_export_models_summary.py +++ b/tests/test_export_models_summary.py @@ -11,7 +11,7 @@ to the operator — they saw a bare ``KeyError`` traceback instead. ``export_models.py`` imports ``tensorrt`` and ``ultralytics_patches`` at -module level (it's a script meant to run inside the ``yolo-api`` container, +module level (it's a script meant to run inside the ``api`` container, which has both), so these tests skip cleanly wherever that stack isn't installed — same convention as ``pytest.importorskip('onnxruntime')`` in ``tests/test_pe_image_encoder_export.py``. diff --git a/tests/test_export_paddleocr_rec.py b/tests/test_export_paddleocr_rec.py index ba63bf028..257080034 100644 --- a/tests/test_export_paddleocr_rec.py +++ b/tests/test_export_paddleocr_rec.py @@ -2,11 +2,11 @@ ``export/export_paddleocr_rec.py``. F-17 (fresh-start E2E findings 2026-09-25): this script runs inside -``yolo-api``, which has no docker CLI or docker.sock, and the old +``api``, which has no docker CLI or docker.sock, and the old ``convert_to_tensorrt_via_trtexec`` shelled out to ``docker exec trtexec ...`` -- always failing with "triton-server container is not running" (the container is renamed -``${COMPOSE_PROJECT_NAME}-triton`` post-G-01, and yolo-api can't run +``${COMPOSE_PROJECT_NAME}-triton`` post-G-01, and api can't run docker commands regardless). ``convert_to_tensorrt_via_python_api`` replaces it with the TensorRT Python API directly (same dependency ``export/export_models.py`` already uses in-process), so these tests diff --git a/tests/test_monitoring_config.py b/tests/test_monitoring_config.py index 35c41b0e4..45d0d887c 100644 --- a/tests/test_monitoring_config.py +++ b/tests/test_monitoring_config.py @@ -91,14 +91,14 @@ def test_only_this_projects_triton_and_api_are_kept(project: str) -> None: blocks = _blocks() env = {'OP_LOG_PROJECT': project} assert _kept(blocks['triton'], _container(project, 'triton-server'), env) - assert _kept(blocks['fastapi'], _container(project, 'yolo-api'), env) + assert _kept(blocks['fastapi'], _container(project, 'api'), env) for other in ('openprocessor', 'opfinal', 'opinst-other'): if other == project: continue # Same suffixes, another stack: must not be collected. assert not _kept(blocks['triton'], _container(other, 'triton-server'), env) - assert not _kept(blocks['fastapi'], _container(other, 'yolo-api'), env) - assert not _kept(blocks['triton'], _container(project, 'yolo-api'), env) + assert not _kept(blocks['fastapi'], _container(other, 'api'), env) + assert not _kept(blocks['triton'], _container(project, 'api'), env) assert not _kept(blocks['fastapi'], _container(project, 'triton-server'), env) assert not _kept(blocks['fastapi'], _container(project, 'curation-detection-worker'), env) @@ -118,7 +118,7 @@ def test_worker_and_segmenter_logs_are_kept_for_this_project_only(project: str) for service in _WORKER_SERVICES: assert _kept(block, _container(project, service), env), service assert not _kept(block, _container('opfinal', service), env), service - for service in ('yolo-api', 'triton-server', 'opensearch', 'curation-cluster-refresh'): + for service in ('api', 'triton-server', 'opensearch', 'curation-cluster-refresh'): assert not _kept(block, _container(project, service), env), service # fullmatch: a look-alike service name must not slip through assert not _kept(block, _container(project, 'segmenter-extra'), env) diff --git a/tests/test_readme_sample_walker_and_docker_test_path.py b/tests/test_readme_sample_walker_and_docker_test_path.py index 282389f75..e0a65748f 100644 --- a/tests/test_readme_sample_walker_and_docker_test_path.py +++ b/tests/test_readme_sample_walker_and_docker_test_path.py @@ -13,8 +13,8 @@ ``OP_SOURCE_ROOT_HOST=./data/samples`` and pass a ``--root`` under ``/data/source/...``. 2. **F-31: the "Docker-only" pytest path doesn't exist.** The production - ``yolo-api`` image installs only ``requirements.txt`` -- no pytest, no - ``requirements-test.txt`` -- so ``docker compose exec yolo-api pytest + ``api`` image installs only ``requirements.txt`` -- no pytest, no + ``requirements-test.txt`` -- so ``docker compose exec api pytest ...`` fails with "executable file not found". README must install ``requirements-test.txt`` into the running container first. @@ -38,7 +38,7 @@ class allowlist means a stock detector's *entire* label space (all 80 def _compose_source_root_mount() -> str: with (REPO_ROOT / 'docker-compose.yml').open() as fh: compose = yaml.safe_load(fh) - volumes = compose['services']['yolo-api']['volumes'] + volumes = compose['services']['api']['volumes'] match = next(v for v in volumes if 'OP_SOURCE_ROOT_HOST' in str(v)) return str(match) diff --git a/tests/test_scrfd_pipeline.py b/tests/test_scrfd_pipeline.py index b018da97a..ba294d924 100755 --- a/tests/test_scrfd_pipeline.py +++ b/tests/test_scrfd_pipeline.py @@ -13,7 +13,7 @@ Prerequisites: - Triton server running with scrfd_10g_bnkps and arcface_w600k_r50 loaded - - yolo-api running on port 4603 + - api running on port 4603 """ import os diff --git a/tests/test_upgrade_remove_orphans.py b/tests/test_upgrade_remove_orphans.py new file mode 100644 index 000000000..086df2b0f --- /dev/null +++ b/tests/test_upgrade_remove_orphans.py @@ -0,0 +1,37 @@ +"""Whole-stack bring-ups must pass ``--remove-orphans``. + +The compose service was renamed ``yolo-api`` -> ``api`` while keeping the same +``container_name``. Without ``--remove-orphans`` the old container (its service +label no longer exists) stays and the new ``api`` container fails with a name +conflict, so every path that starts the stack on an existing install has to +remove orphans. +""" + +from __future__ import annotations + +import re +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[1] + +# file -> regex matching the whole-stack `up` lines that must carry the flag +_WHOLE_STACK_UP = { + 'openprocessor': r'^\s*dc up -d( --remove-orphans)?$', + 'scripts/setup.sh': r'^\s*dc up -d( --remove-orphans)?$', + 'Makefile': r'^\t\$\((DEV_)?COMPOSE\) (--profile monitoring )?up -d( --build)?( --remove-orphans)?$', + 'setup-openprocessor.sh': r'dc up -d( --remove-orphans)?( opensearch)?( \|\||$)', +} + + +def test_whole_stack_up_commands_remove_orphans() -> None: + for path, pattern in _WHOLE_STACK_UP.items(): + rx = re.compile(pattern) + found = [ + ln.strip() + for ln in (ROOT / path).read_text().splitlines() + if rx.search(ln) and not ln.lstrip().startswith('#') + ] + assert found, f'{path}: pattern matched no up command; update this test' + missing = [ln for ln in found if '--remove-orphans' not in ln] + assert not missing, f'{path}: up without --remove-orphans: {missing}'