fix(lab): fence lab Grafana, BloodHound and scratch's UIs to Hicks

docs/security.md

@@ -22,6 +22,7 @@ What this network is actually built to survive:
 | Someone on the lab segment fetching a Velociraptor client package | **Scoped by address, and accepted.** `odin`'s `caddy` service (container `soc-agent-msi`) serves the two agent MSIs `ansible/` installs, and the two Debian packages `garuda` installs, on `8448` ([#1068](https://github.com/Gerrrt/HomeLab/issues/1068), [ADR-0092](adr/0092-enrol-garuda-in-odins-soc-with-debian-packages-staged-on-odin.md)). Its Caddyfile answers the six domain addresses, `.50`–`.55`, and `garuda`'s `.62`, and refuses everyone else, Hicks's route from VLAN 50 included. The Velociraptor MSI and the Velociraptor `.deb` each carry the client config: the server URL, its CA, and the enrolment nonce. Whoever holds either can enrol a rogue client, which is why #864 took the MSI out of SYSVOL. The residual: an address on this segment can be spoofed, and any of the seven can fetch either package. What the nonce buys is noise in Velociraptor's client list, the same class as the Wazuh enrolment password. Rotating it means a new server config, a rebuilt MSI with the role's pin moved, and the staged `.deb` deleted so that `scripts/stage-agent-msis.sh` builds it again. The two Wazuh packages carry no secret |
 | An OpenVAS plugin on `garuda` reaching a non-target | **Fenced at layer 2** ([ADR-0093](adr/0093-run-openvas-on-garuda-on-demand-scoped-to-the-domain-by-nftables.md)). OpenVAS scans the domain's six only. While a scan runs, the scanner runs in a network namespace whose one route to the segment is a macvlan, `scan0` at `10.0.30.63`, behind an nftables `netdev` egress hook that drops every frame — IP, ARP (`send_arp_request`) and raw (`send_frame`) — to anything but the six. So the AF_PACKET paths that bypass an IP filter are caught here. Proved: a GVM scan of an out-of-scope host raised the egress drop counter with 0 open ports, and an `arping` from the namespace to an out-of-scope host was dropped while an in-scope host answered. `gvmd`, `notus`, `redis` and `mosquitto` stay on the host; only MQTT crosses to the namespace, on a veth that routes nowhere else |
 | Someone on the lab segment reaching the SOC's web UIs | **Scoped by address, and accepted.** The Wazuh dashboard (`443`) and the Velociraptor GUI (`8889`) on `odin` are published by the stack's `caddy` service (container `soc-agent-msi`), not by their own containers, and its Caddyfile answers Hicks's `10.0.50.0/24` and refuses everyone else ([#1139](https://github.com/Gerrrt/HomeLab/issues/1139)). Until then both answered all of VLAN 30, the six domain guests included, because traffic between VLAN 30 hosts never crosses pfSense — so no firewall rule could have narrowed them. WireGuard peers are not in the list. The residuals: Hicks is a whole /24 with no per-host source yet ([ADR-0031](adr/0031-narrow-hicks-to-a-named-list-on-winterfell-and-leave-the-lab-open.md)), and an address on VLAN 30 can be spoofed, so this narrows who reaches the login pages and does not replace them. And the hop from Caddy to Velociraptor skips certificate verification: the GUI's certificate is signed by the CA inside the gitignored `server.config.yaml` on `odin`, and the hop is container to container on the `soc` bridge. The dashboard hop is verified against the Wazuh generator's root |
+| Someone on the lab segment reaching the lab's Grafana or BloodHound, or a detonation reaching `diabolos`'s UIs | **Scoped by address, as the SOC's are.** `alexander`'s Grafana (`3000`) is published by the lab stack's `caddy`, and `eden`'s BloodHound (`8443`) by the bloodhound stack's own `caddy`; each answers Hicks's `10.0.50.0/24` and refuses the rest of VLAN 30, the six domain guests included. `diabolos`, not yet built, has soc's fence for its dashboard and GUI. Until then Grafana and BloodHound answered all of VLAN 30, where BloodHound's graph is the domain's attack-path map ([#1428](https://github.com/Gerrrt/HomeLab/issues/1428)). The residuals are the SOC row's: Hicks is a whole /24, and an address on VLAN 30 can be spoofed for a one-way packet but not for the TLS session the UIs need |
 | A range target with a path out | It has none — `gandalf`'s targets sit on a bridge with no physical port, on `172.30.30.0/24`, which the firewall does not route and on which nothing has a default route at all ([ADR-0014](adr/0014-put-ifrit-on-imaginationlan-and-give-the-targets-no-route.md), [ADR-0017](adr/0017-buy-ifrit-for-iops-and-keep-the-range-disposable.md)) |
 | Someone with the trusted Wi-Fi key quietly joining | Kea's lease log reaches Loki; `UnknownDeviceOnTrustedSegment` fires the first time a MAC appears on VLAN 50 in seven days ([ADR-0019](adr/0019-read-device-joins-from-the-dhcp-server.md)) |
 | Losing visibility of a failure | 182 alert rules, 30 days of metrics and logs |

stacks/bloodhound/Caddyfile

@@ -0,0 +1,32 @@
+# BloodHound's UI, answering Hicks only (#1428).
+#
+# It was published on all of VLAN 30, which holds the deliberately weak domain
+# whose attack paths it graphs: a foothold there could read the defenders' map
+# of it. pfSense never sees traffic inside the VLAN, so the source filter has to
+# be here, as soc's is (#1139). BloodHound's own leaf is reused, and the
+# upstream is verified against the estate CA under the `bloodhound` name its
+# certificate carries.
+{
+	admin off
+	auto_https off
+	persist_config off
+}
+
+:8443 {
+	tls /etc/caddy/tls/cert.pem /etc/caddy/tls/key.pem
+	@hicks remote_ip 10.0.50.0/24
+	handle @hicks {
+		reverse_proxy https://bloodhound:8080 {
+			header_up Host {hostport}
+			transport http {
+				tls_trust_pool file /etc/caddy/tls/ca.pem
+				tls_server_name bloodhound
+			}
+		}
+	}
+	respond 403
+}
+
+:2020 {
+	respond /healthz 200
+}

stacks/bloodhound/README.md

@@ -24,7 +24,8 @@ make up STACK=bloodhound
 
 | Service | Image | Port | Purpose |
 | --- | --- | --- | --- |
-| `bloodhound` | `specterops/bloodhound` | 8443 (https) | The API, the ingest workers and the UI. TLS on a leaf from the estate's CA. Serves the SharpHound and AzureHound release it was built with |
+| `bloodhound` | `specterops/bloodhound` | *internal* (https) | The API, the ingest workers and the UI. TLS on a leaf from the estate's CA. Serves the SharpHound and AzureHound release it was built with |
+| `caddy` | `caddy` | 8443 (https) | The UI's fence: answers Hicks's `10.0.50.0/24` only and refuses the rest of VLAN 30, the weak domain included, with BloodHound's own leaf ([#1428](https://github.com/Gerrrt/HomeLab/issues/1428)) |
 | `app-db` | `postgres` | *internal* | Users, saved queries, upload jobs, audit log, **and the graph**: BloodHound's Postgres graph driver, not upstream's default Neo4j ([ADR-0081]) |
 | `alloy` | `grafana/alloy` | 12345 (localhost) | This guest's collector. Pushes to `alexander`, and scrapes BloodHound's metrics as `job="bloodhound"` |
 | `docker-socket-proxy` | `tecnativa/docker-socket-proxy` | *internal* | Holds the Docker socket so Alloy does not: GET-only, the estate's allowlist ([#836](https://github.com/Gerrrt/HomeLab/issues/836)) |

stacks/bloodhound/compose.yaml

@@ -199,12 +199,52 @@ services:
     expose:
       # Prometheus metrics, for alloy/bloodhound.alloy. Not published.
       - "2112"
-    ports:
-      - "${BIND_ADDR:-0.0.0.0}:${BLOODHOUND_PORT:-8443}:8080"
+    # The UI is NOT published (#1428): caddy below publishes the same host
+    # port, answers Hicks's /24 only and proxies here; see the Caddyfile.
     depends_on:
       app-db:
         condition: service_healthy
 
+  # ---------------------------------------------------------------------------
+  # The UI's fence (#1428), soc's #1139 shape: Hicks only, the rest of VLAN 30,
+  # the weak domain included, refused.
+  # ---------------------------------------------------------------------------
+  caddy:
+    image: caddy:2.11.9@sha256:0c2243efcf437ab7f7a978f1027fcf0d4f9be8a75df27b1b3fbc5dbe0ffa8cf0
+    container_name: bloodhound-ui-proxy
+    restart: unless-stopped
+    logging: *default-logging
+    init: true
+    pids_limit: 64
+    networks: [bloodhound]
+    user: "65534:65534"
+    group_add:
+      - "${RENDER_GID:?run make render STACK=bloodhound}"
+    cap_drop: [ALL]
+    cap_add: [NET_BIND_SERVICE]
+    security_opt:
+      - no-new-privileges:true
+    read_only: true
+    tmpfs:
+      - /data:size=1m,uid=65534,gid=65534
+      - /config:size=1m,uid=65534,gid=65534
+      - /tmp:size=1m
+    mem_limit: 64m
+    memswap_limit: 64m
+    volumes:
+      - ./Caddyfile:/etc/caddy/Caddyfile:ro
+      - ../../certificates/bloodhound.matrix.elysium.pem:/etc/caddy/tls/cert.pem:ro
+      - ../../certificates/bloodhound.matrix.elysium-key.pem:/etc/caddy/tls/key.pem:ro
+      - ../../certificates/ca.pem:/etc/caddy/tls/ca.pem:ro
+    ports:
+      - "${BIND_ADDR:-0.0.0.0}:${BLOODHOUND_PORT:-8443}:8443"
+    healthcheck:
+      test: ["CMD", "wget", "--spider", "-q", "http://localhost:2020/healthz"]
+      interval: 30s
+      timeout: 5s
+      retries: 3
+      start_period: 10s
+
   # ---------------------------------------------------------------------------
   # Docker socket proxy (#836), exactly as stacks/sensor has it; that file and
   # stacks/observability/compose.yaml have the reasoning and the allowlist's

stacks/lab/Caddyfile

@@ -120,6 +120,26 @@
 
 # The container's own healthcheck, as on the estate: not published, and it
 # answers without touching an upstream.
+# Grafana, answering Hicks only (#1428). It was published on all of VLAN 30,
+# which holds the deliberately weak domain; pfSense never sees traffic inside
+# the VLAN, so the source filter has to be here, as soc's is (#1139). Grafana's
+# own leaf is reused, and the upstream is verified against the estate CA under
+# the `grafana` name its certificate carries.
+:3443 {
+	tls /etc/caddy/tls/cert.pem /etc/caddy/tls/key.pem
+	@hicks remote_ip 10.0.50.0/24
+	handle @hicks {
+		reverse_proxy https://grafana:3000 {
+			header_up Host {hostport}
+			transport http {
+				tls_trust_pool file /etc/caddy/tls/ca.pem
+				tls_server_name grafana
+			}
+		}
+	}
+	respond 403
+}
+
 :2020 {
 	respond /healthz 200
 }

stacks/lab/README.md

@@ -24,8 +24,8 @@ make up STACK=lab        # from the repository root
 | --- | --- | --- | --- |
 | `prometheus` | `prom/prometheus` | 9090 (localhost) | Metrics store, remote-write receiver, rule evaluation |
 | `loki` | `grafana/loki` | 3100 (localhost) | Log store |
-| `caddy` | `caddy` | 9090, 3100 on 10.0.30.40 | The ingest proxy: odin, phoenix, fenrir, golem, eden and garuda push through it with a token each; apart from the health paths, everything else on the segment gets a 401 ([#834]) |
-| `grafana` | `grafana/grafana` | 3000 (https) | Dashboards, the one service a human opens |
+| `caddy` | `caddy` | 9090, 3100 on 10.0.30.40; 3000 (https) | The ingest proxy: odin, phoenix, fenrir, golem, eden and garuda push through it with a token each; apart from the health paths, everything else on the segment gets a 401 ([#834]). On 3000 it is also Grafana's fence: Hicks's `10.0.50.0/24` only ([#1428](https://github.com/Gerrrt/HomeLab/issues/1428)) |
+| `grafana` | `grafana/grafana` | *internal* (https) | Dashboards, the one service a human opens, through caddy's 3000 |
 | `alloy` | `grafana/alloy` | 12345 (localhost) | Metric and log collection |
 | `docker-socket-proxy` | `tecnativa/docker-socket-proxy` | *internal* | Holds the Docker socket so Alloy does not: GET-only, the estate's allowlist ([#836](https://github.com/Gerrrt/HomeLab/issues/836)) |
 

stacks/lab/compose.yaml

@@ -265,11 +265,20 @@ services:
       INGEST_TOKEN_EDEN: ${INGEST_TOKEN_EDEN:?set in secrets/lab.sops.yaml}
       INGEST_TOKEN_GARUDA: ${INGEST_TOKEN_GARUDA:?set in secrets/lab.sops.yaml}
       INGEST_TOKEN_READER: ${INGEST_TOKEN_READER:?set in secrets/lab.sops.yaml}
+    # Grafana's leaf and key, for the Hicks-only :3443 site (#1428). The key is
+    # 0640 to the checkout owner's group, which Grafana reads it through too.
+    group_add:
+      - "${RENDER_GID:?run make render STACK=lab}"
     volumes:
       - ./Caddyfile:/etc/caddy/Caddyfile:ro
+      - ../../certificates/grafana-lab.matrix.elysium.pem:/etc/caddy/tls/cert.pem:ro
+      - ../../certificates/grafana-lab.matrix.elysium-key.pem:/etc/caddy/tls/key.pem:ro
+      - ../../certificates/ca.pem:/etc/caddy/tls/ca.pem:ro
     ports:
       - "${INGEST_BIND_ADDR:-10.0.30.40}:${PROMETHEUS_PORT:-9090}:9090"
       - "${INGEST_BIND_ADDR:-10.0.30.40}:${LOKI_PORT:-3100}:3100"
+      # Grafana, on the host port it had, for Hicks only (#1428).
+      - "${BIND_ADDR:-0.0.0.0}:${GRAFANA_PORT:-3000}:3443"
     depends_on:
       prometheus:
         condition: service_healthy
@@ -339,8 +348,11 @@ services:
       - ../../certificates/grafana-lab.matrix.elysium-key.pem:/etc/grafana/tls/key.pem:ro
     group_add:
       - "${RENDER_GID:?run make render STACK=lab}"
-    ports:
-      - "${BIND_ADDR:-0.0.0.0}:${GRAFANA_PORT:-3000}:3000"
+    # NOT published (#1428): a port here answered all of VLAN 30, the weak
+    # domain included. caddy publishes the same host port, answers Hicks's /24
+    # only and proxies here over the stack network; see the Caddyfile.
+    expose:
+      - "3000"
     depends_on:
       prometheus:
         condition: service_healthy

stacks/scratch/Caddyfile

@@ -0,0 +1,46 @@
+# The two SOC UIs on diabolos, answering Hicks only (#1428).
+#
+# soc's Caddyfile minus the agent-MSI site: soc fenced its dashboard and
+# Velociraptor GUI behind a Hicks-only Caddy in #1139, and this stack, which
+# is soc's shape, published both on 0.0.0.0 to all of VLAN 30. On the guest
+# where samples are detonated, that put the SIEM's and the DFIR tool's login
+# pages in reach of the sample. pfSense never sees traffic inside VLAN 30, so
+# the source filter has to be here.
+{
+	admin off
+	auto_https off
+	persist_config off
+}
+
+:8443 {
+	tls /etc/caddy/tls/cert.pem /etc/caddy/tls/key.pem
+	@hicks remote_ip 10.0.50.0/24
+	handle @hicks {
+		reverse_proxy https://wazuh.dashboard:5601 {
+			header_up Host {hostport}
+			transport http {
+				tls_trust_pool file /etc/caddy/tls/ca.pem
+				tls_server_name wazuh.dashboard
+			}
+		}
+	}
+	respond 403
+}
+
+:8889 {
+	tls /etc/caddy/tls/cert.pem /etc/caddy/tls/key.pem
+	@hicks remote_ip 10.0.50.0/24
+	handle @hicks {
+		reverse_proxy https://velociraptor:8889 {
+			header_up Host {hostport}
+			transport http {
+				tls_insecure_skip_verify
+			}
+		}
+	}
+	respond 403
+}
+
+:2020 {
+	respond /healthz 200
+}

stacks/scratch/README.md

@@ -44,8 +44,9 @@ changes when an investigation starts or ends.
 | --- | --- | --- | --- |
 | `wazuh.indexer` | `wazuh/wazuh-indexer` | *internal* (9200) | OpenSearch: the store. The same 2 GiB heap and shard ceiling as `odin`, and no delete policy |
 | `wazuh.manager` | `wazuh/wazuh-manager` | 1514, 1515 | For the agents of whatever is under investigation. Enrolment still needs a password, this guest's own |
-| `wazuh.dashboard` | `wazuh/wazuh-dashboard` | 443 (https) | Query it hard |
-| `velociraptor` | `ghcr.io/velocidex/velociraptor-server` | 8000, 8889 (https) | Collection from the endpoints involved, with a CA that dies with the guest |
+| `wazuh.dashboard` | `wazuh/wazuh-dashboard` | *internal*, through caddy's 443 | Query it hard |
+| `velociraptor` | `ghcr.io/velocidex/velociraptor-server` | 8000; 8889 through caddy | Collection from the endpoints involved, with a CA that dies with the guest |
+| `caddy` | `caddy` | 443, 8889 (https) | soc's #1139 fence: the two UIs answer Hicks's `10.0.50.0/24` only, so a detonated sample cannot reach the SIEM's or the DFIR tool's login ([#1428](https://github.com/Gerrrt/HomeLab/issues/1428)) |
 | `wazuh.certs-generator` | `wazuh/wazuh-certs-generator` | — | Behind the `certs` profile: run once per incarnation |
 
 The images and digests are `stacks/soc`'s. Dependabot opens a PR for each

stacks/scratch/compose.yaml

@@ -233,8 +233,11 @@ services:
       - ../soc/wazuh/dashboard/opensearch_dashboards.yml:/usr/share/wazuh-dashboard/config/opensearch_dashboards.yml:ro
       - dashboard-config:/usr/share/wazuh-dashboard/data/wazuh/config
       - dashboard-custom:/usr/share/wazuh-dashboard/plugins/wazuh/public/assets/custom
-    ports:
-      - "${BIND_ADDR:-0.0.0.0}:${WAZUH_DASHBOARD_PORT:-443}:5601"
+    # DIFFERENCE 6: soc's #1139 fence, which this stack had not copied. The
+    # dashboard is not published; the caddy service below publishes the same
+    # host port and answers Hicks's /24 only (#1428).
+    expose:
+      - "5601"
     depends_on:
       wazuh.indexer:
         condition: service_healthy
@@ -280,15 +283,50 @@ services:
       # lab's Prometheus scrapes it; nothing scrapes this guest (ADR-0071), and
       # an unauthenticated listener with no consumer is the ADR-0012 case. The
       # listener still exists inside the container, which the healthcheck uses.
+      # 8889, the GUI, is not published: caddy below is (DIFFERENCE 6).
       - "${BIND_ADDR:-0.0.0.0}:${VELOCIRAPTOR_FRONTEND_PORT:-8000}:8000"
-      - "${BIND_ADDR:-0.0.0.0}:${VELOCIRAPTOR_GUI_PORT:-8889}:8889"
     healthcheck:
       test: ["CMD", "wget", "--spider", "-q", "http://localhost:8003/metrics"]
       interval: 30s
       timeout: 5s
       retries: 3
       start_period: 120s
 
+  # DIFFERENCE 6: soc's caddy without its agent-MSI site. It publishes the two
+  # UIs on the host ports they had, answers Hicks's /24 only, and proxies over
+  # the scratch network; the Caddyfile has the reason (#1428).
+  caddy:
+    <<: *service-defaults
+    image: caddy:2.11.9@sha256:0c2243efcf437ab7f7a978f1027fcf0d4f9be8a75df27b1b3fbc5dbe0ffa8cf0
+    container_name: scratch-ui-proxy
+    user: "1000:1000"
+    pids_limit: 64
+    mem_limit: 128m
+    memswap_limit: 128m
+    cap_drop: [ALL]
+    cap_add: [NET_BIND_SERVICE]
+    security_opt:
+      - no-new-privileges:true
+    read_only: true
+    tmpfs:
+      - /data:size=1m,uid=1000,gid=1000
+      - /config:size=1m,uid=1000,gid=1000
+      - /tmp:size=1m
+    volumes:
+      - ./Caddyfile:/etc/caddy/Caddyfile:ro
+      - ./wazuh/certs/wazuh.dashboard.pem:/etc/caddy/tls/cert.pem:ro
+      - ./wazuh/certs/wazuh.dashboard-key.pem:/etc/caddy/tls/key.pem:ro
+      - ./wazuh/certs/root-ca.pem:/etc/caddy/tls/ca.pem:ro
+    ports:
+      - "${BIND_ADDR:-0.0.0.0}:${WAZUH_DASHBOARD_PORT:-443}:8443"
+      - "${BIND_ADDR:-0.0.0.0}:${VELOCIRAPTOR_GUI_PORT:-8889}:8889"
+    healthcheck:
+      test: ["CMD", "wget", "--spider", "-q", "http://localhost:2020/healthz"]
+      interval: 30s
+      timeout: 5s
+      retries: 3
+      start_period: 10s
+
   # DIFFERENCE 5: no Alloy. odin's pushes its journal, container logs and the
   # indexer's health to the lab's stores on alexander; this guest's would put a
   # detonation's noise into the lab's Loki, with the lab's retention, after the