Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,24 @@ history of this codebase and was never published. This release is `[0.4.1]`.

## [Unreleased]

### Added

- Gateway mode (#93): `OP_GATEWAY_SUBPATHS=true` makes Cropwright's nginx serve
Grafana, Prometheus, OpenSearch Dashboards and MLflow under `/grafana/`,
`/prometheus/`, `/dashboards/` and `/mlflow/` of its one published port, so the UIs
stay on `127.0.0.1`. One switch in `docker-compose.yml` starts each UI serving from
its sub-path, the API serves the path-relative `resource_links` and MLflow run
links (an explicit `OP_*_URL` still wins and a `0` port still hides the link), and
nginx upgrades Grafana Live websockets and forwards `Host` and `X-Forwarded-*`.
Fail closed: the UIs refuse to start when published beyond loopback, and Grafana
refuses the default or an empty admin password. Prometheus and MLflow still have no
authentication (`SECURITY.md`). The link audit is in the monitoring guide.

### Fixed

- Cropwright's nginx now answers redirects with relative `Location` headers; the
`/OpenProcessor` redirect used to name the container port 8080.

### Changed

- The experiment-tracking server and the trainer client move to MLflow 3.17 together
Expand Down
4 changes: 4 additions & 0 deletions INSTALLATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,10 @@ everything on 127.0.0.1.**
`0.0.0.0`, so LAN browsers can follow resource links while the datastore and
model APIs stay local. Change `GF_SECURITY_ADMIN_PASSWORD` first; MLflow and
Prometheus have no auth.
- `OP_GATEWAY_SUBPATHS=true` (in `.env`, with the `cropwright` profile) serves the
four monitoring UIs under `/grafana/`, `/prometheus/`, `/dashboards/` and
`/mlflow/` on Cropwright's one port, so the UIs can stay on loopback; it needs a
non-default `GF_SECURITY_ADMIN_PASSWORD`. See the monitoring guide.
- A re-run keeps the Cropwright bind you chose last time.

Details: [SECURITY.md](SECURITY.md).
Expand Down
14 changes: 13 additions & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,19 @@ the files it covers. Release signing is follow-up work.
- **Grafana** ships with the default credentials `admin` / `admin`
(`GF_SECURITY_ADMIN_USER` / `GF_SECURITY_ADMIN_PASSWORD` in
`docker-compose.yml`). Change these before exposing Grafana to
anyone but yourself.
anyone but yourself. In gateway mode (`OP_GATEWAY_SUBPATHS=true`) Grafana
refuses to start with the default or an empty password. The variable only
sets the password when the `grafana_data` volume is first created; on an
existing volume change it with
`docker compose exec grafana grafana cli admin reset-admin-password <new>`.
- **Prometheus and MLflow** have no authentication at all. In gateway mode
(see [the monitoring guide](docs-site/docs/operations/monitoring.mdx)) anyone
who can reach the gateway port can read Prometheus and read and write MLflow
experiments and registered models, exactly as they can already use the API. The
gateway moves these from "one port each" to "behind the one gateway port"; it
adds no login. Publish the gateway only on a trusted network or behind an
authenticating reverse proxy. Gateway mode also refuses to start the four UIs
when they are published beyond loopback.
- **OpenSearch** ships with its security plugin disabled by default
(`DISABLE_SECURITY_PLUGIN=true` / `DISABLE_SECURITY_DASHBOARDS_PLUGIN=true`
in `docker-compose.yml`, commented "Disable security for dev (enable
Expand Down
77 changes: 76 additions & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,54 @@ x-logging: &default-logging
max-size: "10m"
max-file: "5"

# Gateway mode (OP_GATEWAY_SUBPATHS=true): the Cropwright nginx serves Grafana,
# Prometheus, OpenSearch Dashboards and MLflow under /grafana/, /prometheus/,
# /dashboards/ and /mlflow/ of its one published port. Each UI must then serve
# FROM its sub-path, which Grafana and OpenSearch Dashboards take as env and
# Prometheus as flags (MLflow is left rooted and the gateway strips /mlflow:
# --static-prefix would also prefix /health and /api and break the in-network
# MLFLOW_TRACKING_URI). All four start through this one wrapper so the safety
# checks below apply to each: `sh -c <script> gateway-wrap <service> <real
# entrypoint...>`, with the service's `command` appended. Off (the default) it
# is a plain exec.
# Fail closed: gateway mode refuses to start a UI that is published beyond
# loopback (OP_UI_BIND_ADDRESS / OP_BIND_ADDRESS), and Grafana with the default
# or an empty admin password.
# The entrypoints restated next to each use (/run.sh, /bin/prometheus,
# ./opensearch-dashboards-docker-entrypoint.sh, none for MLflow) must match each
# image's own ENTRYPOINT. Re-check with
# docker image inspect --format '{{json .Config.Entrypoint}}' <image>
# whenever the grafana, prometheus, opensearch-dashboards or mlflow image is bumped.
x-gateway-wrap: &gateway_wrap |
tag="$$1"; shift
case "$$(printf '%s' "$${OP_GATEWAY_SUBPATHS:-}" | tr 'A-Z' 'a-z')" in
1|true|yes|on) ;;
''|0|false|no|off) exec "$$@" ;;
*) echo "gateway-wrap: OP_GATEWAY_SUBPATHS must be true or false, got: $$OP_GATEWAY_SUBPATHS" >&2; exit 1 ;;
esac
case "$${OP_UI_PUBLISH_ADDRESS:-}" in
127.*|localhost|::1|'[::1]') ;;
*) echo "gateway-wrap: refusing to start $$tag: OP_GATEWAY_SUBPATHS=true but the UI ports are published on '$$OP_UI_PUBLISH_ADDRESS'. Unset OP_UI_BIND_ADDRESS (or set it to 127.0.0.1) and publish only the gateway (CROPWRIGHT_BIND_ADDRESS)." >&2; exit 1 ;;
esac
case "$$tag" in
grafana)
case "$${GF_SECURITY_ADMIN_PASSWORD:-}" in
''|admin) echo "gateway-wrap: refusing to start grafana behind the gateway with the default admin password. Set GF_SECURITY_ADMIN_PASSWORD to a non-default value (an existing grafana_data volume keeps its old password: see SECURITY.md)." >&2; exit 1 ;;
esac
export GF_SERVER_ROOT_URL='%(protocol)s://%(domain)s/grafana/'
export GF_SERVER_SERVE_FROM_SUB_PATH=true
exec "$$@" ;;
prometheus)
exec "$$@" --web.external-url=/prometheus/ --web.route-prefix=/prometheus/ ;;
dashboards)
export SERVER_BASEPATH=/dashboards
export SERVER_REWRITEBASEPATH=true
exec "$$@" ;;
mlflow)
exec "$$@" ;;
esac
echo "gateway-wrap: unknown service tag $$tag" >&2; exit 1

services:
triton-server:
image: ${OP_TRITON_IMAGE:-${OP_IMAGE_REPO:-davidamacey}/openprocessor-triton:${OP_IMAGE_TAG:-0.4.1}}
Expand Down Expand Up @@ -153,6 +201,8 @@ services:
# builds its prefix at *module import*, so this must be set before
# src.main is imported; it cannot change at container runtime.
- OP_API_PREFIX=${OP_API_PREFIX:-/curation}
# Gateway mode: served resource_links are path-relative (/grafana/ ...).
- OP_GATEWAY_SUBPATHS=${OP_GATEWAY_SUBPATHS:-false}
# G-06: in-container path the API resolves source-relative item
# paths against. Must match the bind target above.
- OP_SOURCE_ROOT=${OP_SOURCE_ROOT:-/data/source}
Expand Down Expand Up @@ -256,7 +306,9 @@ services:
- op-api

# ===================================================================
# Cropwright: the labeling frontend, opt-in via `--profile cropwright`.
# Cropwright: the labeling frontend, opt-in via `--profile cropwright`. It is
# also the gateway (OP_GATEWAY_SUBPATHS=true): the one published port for the
# app, the API docs and the monitoring UIs, which then stay on loopback.
# Source lives in ./frontend; nginx proxies the API prefix to yolo-api by
# its op-api alias, so the browser only ever talks to this origin.
# ===================================================================
Expand Down Expand Up @@ -290,6 +342,9 @@ services:
- CROPWRIGHT_DATASET_UPLOAD_MAX_MB=${CROPWRIGHT_DATASET_UPLOAD_MAX_MB:-2048}
# Where nginx proxies /OpenProcessor/ (the docs site): the docs service of docker-compose.docs.yml, when run.
- DOCS_UPSTREAM=${DOCS_UPSTREAM:-http://docs:8080}
# Gateway mode: also serve the monitoring UIs under /grafana/, /prometheus/,
# /dashboards/ and /mlflow/ of this one origin (see x-gateway-wrap below).
- OP_GATEWAY_SUBPATHS=${OP_GATEWAY_SUBPATHS:-false}
ports:
# A UI: CROPWRIGHT_BIND_ADDRESS opens just this one to a LAN (the installer's default for
# homelabs), OP_UI_BIND_ADDRESS opens it with the other UIs, and the API stays on loopback.
Expand Down Expand Up @@ -705,6 +760,7 @@ services:
profiles:
- training
restart: unless-stopped
entrypoint: ['/bin/sh', '-c', *gateway_wrap, 'gateway-wrap', 'mlflow']
command:
# ML-1: serve artifacts through the tracking server's HTTP proxy
# instead of a bare local path. The trainer container never mounts
Expand All @@ -728,6 +784,8 @@ services:
# localhost + private-IP default, so those are restated here next to
# the in-network names the trainer and API use. Add the DNS name you
# browse to (MLFLOW_ALLOWED_HOSTS replaces the whole list).
- OP_GATEWAY_SUBPATHS=${OP_GATEWAY_SUBPATHS:-false}
- OP_UI_PUBLISH_ADDRESS=${OP_UI_BIND_ADDRESS:-${OP_BIND_ADDRESS:-127.0.0.1}}
- MLFLOW_SERVER_ALLOWED_HOSTS=${MLFLOW_ALLOWED_HOSTS:-curation-mlflow,curation-mlflow:*,${COMPOSE_PROJECT_NAME:-openprocessor}-mlflow,${COMPOSE_PROJECT_NAME:-openprocessor}-mlflow:*,localhost,localhost:*,127.0.0.1,127.0.0.1:*,[::1],[[]::1]:*,192.168.*,10.*,172.16.*,172.17.*,172.18.*,172.19.*,172.20.*,172.21.*,172.22.*,172.23.*,172.24.*,172.25.*,172.26.*,172.27.*,172.28.*,172.29.*,172.30.*,172.31.*}
volumes:
- mlflow_data:/mlflow
Expand Down Expand Up @@ -893,6 +951,10 @@ services:
restart: always
profiles:
- monitoring
entrypoint: ['/bin/sh', '-c', *gateway_wrap, 'gateway-wrap', 'prometheus', '/bin/prometheus']
environment:
- OP_GATEWAY_SUBPATHS=${OP_GATEWAY_SUBPATHS:-false}
- OP_UI_PUBLISH_ADDRESS=${OP_UI_BIND_ADDRESS:-${OP_BIND_ADDRESS:-127.0.0.1}}
ports:
- "${OP_UI_BIND_ADDRESS:-${OP_BIND_ADDRESS:-127.0.0.1}}:${PROMETHEUS_PORT:-4604}:9090" # Prometheus
volumes:
Expand All @@ -918,6 +980,7 @@ services:
restart: always
profiles:
- monitoring
entrypoint: ['/bin/sh', '-c', *gateway_wrap, 'gateway-wrap', 'grafana', '/run.sh']
ports:
- "${OP_UI_BIND_ADDRESS:-${OP_BIND_ADDRESS:-127.0.0.1}}:${GRAFANA_PORT:-4605}:3000" # Grafana
volumes:
Expand All @@ -933,6 +996,8 @@ services:
# Enable unified alerting (Grafana 8+)
- GF_UNIFIED_ALERTING_ENABLED=true
- GF_ALERTING_ENABLED=false
- OP_GATEWAY_SUBPATHS=${OP_GATEWAY_SUBPATHS:-false}
- OP_UI_PUBLISH_ADDRESS=${OP_UI_BIND_ADDRESS:-${OP_BIND_ADDRESS:-127.0.0.1}}
networks:
- triton_net
depends_on:
Expand Down Expand Up @@ -1027,7 +1092,17 @@ services:
restart: always
profiles:
- monitoring
entrypoint:
- /bin/sh
- -c
- *gateway_wrap
- gateway-wrap
- dashboards
- ./opensearch-dashboards-docker-entrypoint.sh
command: ['opensearch-dashboards']
environment:
- OP_GATEWAY_SUBPATHS=${OP_GATEWAY_SUBPATHS:-false}
- OP_UI_PUBLISH_ADDRESS=${OP_UI_BIND_ADDRESS:-${OP_BIND_ADDRESS:-127.0.0.1}}
- OPENSEARCH_HOSTS=["http://opensearch:9200"]
- DISABLE_SECURITY_DASHBOARDS_PLUGIN=true # Must match OpenSearch security setting
ports:
Expand Down
50 changes: 48 additions & 2 deletions docs-site/docs/operations/monitoring.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -217,14 +217,17 @@ reachable}`.
1. `OP_GRAFANA_URL`, `OP_PROMETHEUS_URL`, `OP_DASHBOARDS_URL`
or `OP_MLFLOW_PUBLIC_URL`, served
verbatim. Use these behind a reverse proxy or a DNS name.
2. Otherwise `<scheme>://<host the client used>:<service port>`. The host
2. With `OP_GATEWAY_SUBPATHS=true` (see [Gateway mode](#gateway-mode-one-port)),
the path-relative `/grafana/`, `/prometheus/`, `/dashboards/` and `/mlflow/`.
A port of `0` still hides the link.
3. Otherwise `<scheme>://<host the client used>:<service port>`. The host
comes from `X-Forwarded-Host` (first value) or `Host`, the scheme from
`X-Forwarded-Proto` or the connection. A LAN client opening
`http://10.10.10.20:5184` gets `http://10.10.10.20:<port>`, never
`localhost`. The ports are `OP_GRAFANA_PORT` (4605), `OP_PROMETHEUS_PORT`
(4604), `OP_DASHBOARDS_PORT` (4608), and `OP_MLFLOW_PORT` (4609); `0` disables.
`make dev-up` passes the stack's own `*_PORT` values.
3. No request context, or port `0`: `url: null`, `status: not_configured`.
4. No request context, or port `0`: `url: null`, `status: not_configured`.
- The MLflow run link (`mlflow_run_url` on train status and manifest, and
`mlflow_public_url` on `/health`) follows the same rule.
- `reachable` means the service answers from the server: the API probes the
Expand All @@ -244,6 +247,49 @@ port from the published port. Security: Grafana ships `admin`/`admin` (set
`GF_SECURITY_ADMIN_PASSWORD` before exposing it); MLflow and Prometheus have no auth. Behind a single-origin reverse
proxy, set the `OP_*_URL` overrides to the proxied addresses instead.

## Gateway mode (one port)

Instead of publishing a port per UI, publish only Cropwright's port and let its
nginx proxy the UIs under sub-paths. The UIs stay on `127.0.0.1`.

```bash
# .env
COMPOSE_PROFILES=cropwright,monitoring,training
OP_GATEWAY_SUBPATHS=true
GF_SECURITY_ADMIN_PASSWORD=<not admin>
CROPWRIGHT_BIND_ADDRESS=0.0.0.0 # the gateway; leave OP_UI_BIND_ADDRESS unset
```

One switch does everything: compose starts Grafana, Prometheus and OpenSearch
Dashboards serving from their sub-path, Cropwright's nginx adds the routes, and
the API serves the path-relative links. MLflow keeps its own root (the gateway
strips `/mlflow`) so the trainer's in-network tracking URI is unchanged, and the
gateway sends MLflow its own service name as `Host`, so its allowed-hosts guard
passes whatever name or address the browser used. nginx forwards `Host` and
`X-Forwarded-Host` (with the port) and `X-Forwarded-Proto` on every proxied
location; Grafana Live (`/grafana/api/live/`) is upgraded to a websocket.

Fail closed: with the switch on, the four UI services refuse to start if
`OP_UI_BIND_ADDRESS` / `OP_BIND_ADDRESS` publish them beyond loopback, and
Grafana refuses the default or an empty `GF_SECURITY_ADMIN_PASSWORD`. Prometheus
and MLflow have no login: anyone who reaches the gateway port reaches them (see
`SECURITY.md`). The gateway only proxies to bare compose service names
(`GRAFANA_UPSTREAM`, `PROMETHEUS_UPSTREAM`, `DASHBOARDS_UPSTREAM`,
`MLFLOW_UPSTREAM` override the defaults; an IP, dotted name or path is refused).

### Link audit

| Link | Gateway URL | Notes |
| --- | --- | --- |
| Swagger UI | `/docs` | Proxied to the API at the root, with `/docs/oauth2-redirect` |
| ReDoc | `/redoc` | Assets from `/docs-assets/` |
| OpenAPI JSON | `/openapi.json` | |
| Grafana | `/grafana/` | `GF_SERVER_SERVE_FROM_SUB_PATH`; websocket on `/grafana/api/live/`; login required |
| Prometheus | `/prometheus/` | `--web.external-url` and `--web.route-prefix` `/prometheus/`; no auth |
| OpenSearch Dashboards | `/dashboards/` | `server.basePath` and `server.rewriteBasePath`; no auth |
| MLflow | `/mlflow/` | Prefix stripped by the gateway; run links are `/mlflow/#/experiments/<id>/runs/<run>`; no auth |
| Triton metrics, DCGM | not routed | Stay on their own loopback ports; no resource link when `OP_*_PORT` is 0 |

`make dev-up` starts the dev stack; the monitoring profile is opt-in
(`make dev-monitoring-up` adds it), as DCGM reserves every GPU and Alloy reads
the Docker socket.
Expand Down
19 changes: 18 additions & 1 deletion env.template
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,22 @@ OPENSEARCH_HEAP=2g
# a LAN address only on a network you trust.
# CROPWRIGHT_BIND_ADDRESS=
# CROPWRIGHT_PORT=5184
#
# Gateway mode (OP_GATEWAY_SUBPATHS): Cropwright's nginx also serves the four
# monitoring UIs under /grafana/, /prometheus/, /dashboards/ and /mlflow/ of its
# one published port (CROPWRIGHT_PORT), so only that port needs to be reachable
# and the UIs stay published on 127.0.0.1 only. Requires the `cropwright`
# profile. One switch for the whole stack: compose reads it for Cropwright
# (which adds the routes), Grafana, Prometheus, OpenSearch Dashboards and MLflow
# (which then serve from their sub-path), and the API reads it so the served
# resource_links are the path-relative /grafana/ ... instead of
# <host>:<port> (an explicit OP_*_URL below still wins; a 0 OP_*_PORT still
# hides that link). Fail closed: with it on, those four services refuse to
# start when OP_UI_BIND_ADDRESS / OP_BIND_ADDRESS publish them beyond loopback,
# and Grafana refuses the default or an empty GF_SECURITY_ADMIN_PASSWORD.
# Prometheus and MLflow have no login of their own: anyone who reaches the
# gateway port reaches them (SECURITY.md).
# OP_GATEWAY_SUBPATHS=false

# =============================================================================
# Isolation (G-01) -- run a second, independent stack on the same host
Expand Down Expand Up @@ -662,7 +678,8 @@ VLM_PORT=4612
# <the host the browser used>:<published port>, so LAN clients get their own
# host. The ports follow OP_<X>_PORT (compose defaults; 0 disables; `make
# dev-up` passes the stack's *_PORT values). Set OP_<X>_URL only to override
# verbatim, e.g. behind a reverse proxy. API docs links are path-relative.
# verbatim, e.g. behind a reverse proxy. API docs links are path-relative, and
# with OP_GATEWAY_SUBPATHS=true so are the four service links.
# OP_GRAFANA_PORT=4605
# OP_PROMETHEUS_PORT=4604
# OP_DASHBOARDS_PORT=4608
Expand Down
11 changes: 11 additions & 0 deletions frontend/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- Gateway mode: with `OP_GATEWAY_SUBPATHS=true` the entrypoint installs
`nginx-gateway.conf`, which proxies `/grafana/`, `/prometheus/`, `/dashboards/` and
`/mlflow/` to the compose services (bare service names only, overridable with
`GRAFANA_UPSTREAM`, `PROMETHEUS_UPSTREAM`, `DASHBOARDS_UPSTREAM`, `MLFLOW_UPSTREAM`),
with the websocket upgrade for Grafana Live. No ports are hardcoded in the app.
With the switch off those four paths answer 404 instead of the app shell, and an
`https` `X-Forwarded-Proto` from an outer TLS proxy is kept on the gateway locations.
- The item wire carries `detector_class_name`, `detector_class_id` and `detector_confidence` (the detector's own class, kept next to the VLM or human label); `RawCrop` and the test fixtures list them.

### Fixed

- nginx uses `absolute_redirect off`, so redirects no longer point at container port 8080.

### Changed

- Moved to SvelteKit 3 and `@sveltejs/adapter-static` 4 on TypeScript 6. SvelteKit
Expand Down
10 changes: 10 additions & 0 deletions frontend/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2712,6 +2712,16 @@ resets and re-reads on a project switch). Tests: `resourceLinks.test.ts`,
`contract/curationSettingsContract.test.ts`, the nginx cases in
`entrypoint.test.ts`, e2e `test_resources_menu.py`.

### Gateway mode (OpenProcessor #93)

`OP_GATEWAY_SUBPATHS=true` (read by `docker-entrypoint.sh`, same spelling as the
API's) copies `nginx-gateway.conf` into `/etc/nginx/gateway.d/`, which `nginx.conf`
includes; off, the directory is empty. It proxies `/grafana/`, `/prometheus/`,
`/dashboards/` and `/mlflow/` (MLflow with `/mlflow` stripped and its own service
name as `Host`; Grafana Live upgraded) to bare compose service names, validated by
the entrypoint. The app needs no code for it: the served `resource_links` become
path-relative and `safeHref` already accepts them. Tests: `gatewayConf.test.ts`.

### Projects — `/p/[project]` routes, switcher, `/projects`

**OWNER DECISION: a fresh build, no backward compatibility.** The
Expand Down
6 changes: 4 additions & 2 deletions frontend/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,14 @@ RUN apk upgrade --no-cache && apk del --no-cache curl && rm -rf /usr/share/nginx
COPY --from=builder /app/build /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.conf
COPY nginx-security-headers.conf /etc/nginx/snippets/security-headers.conf
COPY nginx-gateway.conf /etc/nginx/snippets/gateway-subpaths.conf
COPY docker-entrypoint.sh /docker-entrypoint.d/40-runtime-config.sh
# The runtime-config script rewrites placeholders in the bundle and in
# default.conf with `sed -i` at container start, as uid 101.
RUN chmod 0755 /docker-entrypoint.d/40-runtime-config.sh \
&& chown -R 101:0 /usr/share/nginx/html /etc/nginx/conf.d \
&& chmod -R g+w /usr/share/nginx/html /etc/nginx/conf.d
&& mkdir -p /etc/nginx/gateway.d \
&& chown -R 101:0 /usr/share/nginx/html /etc/nginx/conf.d /etc/nginx/gateway.d \
&& chmod -R g+w /usr/share/nginx/html /etc/nginx/conf.d /etc/nginx/gateway.d
USER 101

EXPOSE 8080
Expand Down
Loading
Loading