diff --git a/.github/workflows/fast-path.yml b/.github/workflows/fast-path.yml index 8ed45878..b83686d4 100644 --- a/.github/workflows/fast-path.yml +++ b/.github/workflows/fast-path.yml @@ -705,6 +705,7 @@ jobs: python3 -m pip install -r ci/agent-relay/kind/requirements.txt make local-acceptance-test PYTHON=python3 python3 ci/agent-relay/test_control_webapp_names.py -v + python3 ci/agent-relay/test_eks_alb_ingress.py -v python3 ci/agent-relay/test_release_candidate.py -v python3 ci/agent-relay/test_release_candidate_gate.py -v python3 ci/agent-relay/test_install_pack.py -v diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index 8c4b4961..3cdd587c 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -126,6 +126,7 @@ jobs: python3 -m pip install -r ci/agent-relay/kind/requirements.txt make local-acceptance-test PYTHON=python3 python3 ci/agent-relay/test_control_webapp_names.py -v + python3 ci/agent-relay/test_eks_alb_ingress.py -v python3 ci/agent-relay/test_release_candidate.py -v python3 ci/agent-relay/test_release_candidate_gate.py -v python3 ci/agent-relay/test_install_pack.py -v diff --git a/charts/agent-relay/Chart.yaml b/charts/agent-relay/Chart.yaml index 7828c1c4..f1b515a2 100644 --- a/charts/agent-relay/Chart.yaml +++ b/charts/agent-relay/Chart.yaml @@ -2,7 +2,7 @@ apiVersion: v2 name: agent-relay description: Self-hosted Agent Relay messaging and sessions suite type: application -version: 0.7.63 +version: 0.7.66 appVersion: "0.1.0" kubeVersion: ">=1.28.0-0" annotations: diff --git a/charts/agent-relay/README.md b/charts/agent-relay/README.md index 8ed566c4..7f980dc7 100644 --- a/charts/agent-relay/README.md +++ b/charts/agent-relay/README.md @@ -396,6 +396,88 @@ read paths to `agent-relay-read-api`, and refuses all other History-host paths at the ALB. Set the script's Service overrides only when the private overlay renames those Services. +### EKS ALB Ingress + +The script stays the documented way to publish the EKS edge. The alternative is +to render the same Ingress from this release, so an install's edge is applied +and versioned with it: set `eksIngress.enabled=true` (default false). For the +same options the chart's Ingress equals the script's output (annotations and +spec, rule order and `pathType` included; the chart only adds +`app.kubernetes.io/name` and `app.kubernetes.io/instance` labels), which +`ci/agent-relay/test_eks_alb_ingress.py` proves by running both. The Ingress +goes into the release namespace. `eksIngress` and `gkeIngress` are exclusive. + +```yaml +eksIngress: + enabled: true + domain: relay.example.com + certificateArn: arn:aws:acm:us-east-1:111111111111:certificate/00000000-0000-0000-0000-000000000000 +``` + +| Option | Script variable | Default | +| --- | --- | --- | +| `eksIngress.name` | `EKS_ALB_INGRESS_NAME` | `agent-relay` | +| `eksIngress.className` | `EKS_INGRESS_CLASS` | `alb` | +| `eksIngress.domain` | `EKS_ALB_DOMAIN` | required | +| `eksIngress.certificateArn` | `EKS_ALB_CERT_ARN` | required | +| `eksIngress.history.enabled` | `EKS_ALB_HISTORY` | `true` | +| `eksIngress.history.readOnly` | `EKS_ALB_HISTORY_READ_ONLY` | `false` | +| `eksIngress.ingest.enabled` | `EKS_ALB_INGEST` | `null`: on when the history host is on and not read-only | +| `eksIngress.ingest.serviceName`, `servicePort` | `EKS_ALB_INGEST_SERVICE`, `_PORT` | `agent-relay-ingest-edge`, `80` | +| `eksIngress.readApi.serviceName`, `servicePort` | `EKS_ALB_READ_API_SERVICE`, `_PORT` | `agent-relay-read-api`, `80` | +| `eksIngress.flows.enabled`, `serviceName`, `servicePort` | `EKS_ALB_RELAYFILE_CONTROL`, `_SERVICE`, `_PORT` | `false`, `relayfile-control`, `3100` | +| `eksIngress.chief.serviceName`, `servicePort` | `EKS_ALB_CHIEF_SERVICE`, `_PORT` | empty (off), `3200` | +| `eksIngress.control.serviceName` | `EKS_ALB_CONTROL_SERVICE` | `relay-control` | +| `eksIngress.webapp.serviceName` | `EKS_ALB_WEBAPP_SERVICE` | `relay-webapp` | +| `eksIngress.controlPaths` | none | `[/cloud/api, /login, /relay-node]` | + +The render refuses what the script refuses: no domain or certificate; read-only +without the history host or with ingest on; ingest off on a history host that is +not read-only; ingest on without the history host; Service names that are not +DNS labels and ports outside 1-65535 (each checked only when its route is +rendered). It also refuses non-boolean switches, a domain that is not a DNS +name, a certificate that is not an ARN, and malformed `controlPaths`. + +`controlPaths` is the list of dashboard-host (``) paths sent to the +control API ahead of the dashboard catch-all, in order (Prefix paths; with Chief +on, each becomes Exact plus `/*` after the Chief wildcards). The script has +no such option; the default is the list it emits. Pin the list your live ALB has +so that a chart upgrade which changes the default does not change your +listener rules unannounced. + +Backends are named, never looked up and never required to be part of this +release: Flows (relayfile-control) and Chief are separate releases, and a +read-only history host may name a read API Service this release does not +install. A wrong name renders and the ALB answers 503 for that route. + +#### Adopting an Ingress that was applied with the script + +Helm will not take over an existing object that lacks its ownership metadata; +the upgrade fails with `invalid ownership metadata`. To move an install from the +script to `eksIngress` without a new load balancer: + +1. Set the `eksIngress` options to the values the script ran with (same + `name`, class, domain, certificate, Service overrides; pin `controlPaths` to + the live list). +2. Render and diff against the live object: + `helm template charts/agent-relay -n -f --show-only templates/eks-alb-ingress.yaml > eks-ingress.yaml`, + then `kubectl diff -f eks-ingress.yaml`. Only the two chart labels may + differ. +3. Add the three ownership fields to the live Ingress: + + ```sh + kubectl -n annotate ingress \ + meta.helm.sh/release-name= meta.helm.sh/release-namespace= + kubectl -n label ingress app.kubernetes.io/managed-by=Helm + ``` + +4. `helm upgrade` with `eksIngress.enabled=true`. + +The same Ingress name and no `alb.ingress.kubernetes.io/group.name` mean the +controller reconciles the same object onto the same load balancer, so its DNS +name and the records pointing at it stay. Stop running the script for that +install afterwards: a later `kubectl apply` of its output would fight Helm. + Install into `agent-relay-` (or another namespace explicitly bound in Terraform). EKS Pod Identity associations must name that namespace and these exact Service Accounts: diff --git a/charts/agent-relay/ci/eks-ingress-values.yaml b/charts/agent-relay/ci/eks-ingress-values.yaml new file mode 100644 index 00000000..91b6a4cd --- /dev/null +++ b/charts/agent-relay/ci/eks-ingress-values.yaml @@ -0,0 +1,23 @@ +# The EKS ALB edge rendered from the release (eksIngress) on the proof shape, +# with every route family on: history (ingest), Flows and Chief. Layered on +# values-gcp-proof.yaml by scripts/ci/charts.sh (lint --strict, kubeconform); +# ci/agent-relay/test_eks_alb_ingress.py proves equality with the script. +gkeIngress: + enabled: false +eksIngress: + enabled: true + domain: relay.example.com + certificateArn: arn:aws:acm:us-east-1:111111111111:certificate/00000000-0000-0000-0000-000000000000 + flows: + enabled: true + chief: + serviceName: relaychief-control +# Render-only identities as in gcp-proof-values.yaml. +relay-control: + image: + digest: sha256:2222222222222222222222222222222222222222222222222222222222222222 + encryption: + keyId: proof-1 +relay-webapp: + image: + digest: sha256:9bfc0c6b873411c7a0da60d746883bdef6aafd86a9b733c719500a603890b6fe diff --git a/charts/agent-relay/templates/_helpers.tpl b/charts/agent-relay/templates/_helpers.tpl index 3a043d53..4890f89b 100644 --- a/charts/agent-relay/templates/_helpers.tpl +++ b/charts/agent-relay/templates/_helpers.tpl @@ -12,6 +12,7 @@ {{- if and (kindIs "map" $brand) (or $brand.svg $brand.existingConfigMap) (dig "control" true (.Values.components | default dict)) (not (dig "webapp" true (.Values.components | default dict))) (not (dig "login" "logoUrl" "" (index .Values "relay-control" | default dict))) }} {{- fail "global.branding.logo is hosted by relay-webapp, but components.webapp=false: the sign-in page would link a logo nothing serves. Enable components.webapp, or host the logo yourself and set relay-control.login.logoUrl" }} {{- end }} +{{- if and (eq (dig "enabled" false (.Values.eksIngress | default dict) | toString) "true") (dig "enabled" false (.Values.gkeIngress | default dict)) }}{{ fail "eksIngress.enabled and gkeIngress.enabled are exclusive: an install has one edge" }}{{ end }} {{- include "agent-relay.validateRemovedHistory" . }} {{- include "agent-relay.validateDurableActors" . }} {{- include "agent-relay.validateSingleHost" . }} @@ -274,3 +275,15 @@ can't derive for its children: the Control Service relaygate pulls from. */ -}} {{- if ne (int (dig "controlServicePort" 80 $cp)) (int (dig "service" "port" 80 $control)) }}{{ fail (printf "global.relaygate.controlPolicy.controlServicePort must equal relay-control.service.port (%v)" (dig "service" "port" 80 $control)) }}{{ end -}} {{- end -}} {{- end }} + +{{- /* "true" when .name is a lowercase DNS name (RFC 1123 subdomain): at most +.max characters, each dot-separated label 1-63 letters, digits and inner +hyphens, so no empty label ("a..b") and no label starting or ending with "-". */ -}} +{{- define "agent-relay.isDnsName" -}} +{{- $name := toString .name -}} +{{- $ok := and $name (le (len $name) (int .max)) -}} +{{- range $l := splitList "." $name }} +{{- if not (and (le (len $l) 63) (regexMatch "^[a-z0-9]([-a-z0-9]*[a-z0-9])?$" $l)) }}{{ $ok = false }}{{ end -}} +{{- end -}} +{{- if $ok }}true{{ end -}} +{{- end }} diff --git a/charts/agent-relay/templates/eks-alb-ingress.yaml b/charts/agent-relay/templates/eks-alb-ingress.yaml new file mode 100644 index 00000000..a00ac6aa --- /dev/null +++ b/charts/agent-relay/templates/eks-alb-ingress.yaml @@ -0,0 +1,256 @@ +{{- /* Checked before the gate: a string "false" is truthy in a template and +would otherwise render the edge. */ -}} +{{- $eksEnabled := dig "enabled" false (.Values.eksIngress | default dict) -}} +{{- if not (kindIs "bool" $eksEnabled) }}{{ fail "eksIngress.enabled must be true or false" }}{{ end -}} +{{- if $eksEnabled }} +{{- /* The EKS ALB edge as a release object: the same Ingress that +scripts/selfhost/eks-alb-ingress.sh prints, option for option (README: EKS ALB +Ingress). The script stays the documented way; ci/agent-relay/test_eks_alb_ingress.py +holds the two to deep equality. Every backend is named, never looked up: +Flows and Chief are separate releases, and a read-only history host may name +a read API Service this release does not install, so nothing here requires a +component of this release to be enabled. */ -}} +{{- $e := .Values.eksIngress -}} +{{- include "agent-relay.validate" . -}} +{{- $label := "^[a-z0-9]([-a-z0-9]*[a-z0-9])?$" -}} +{{- $port := "^[1-9][0-9]{0,4}$" -}} +{{- $bool := dict -}} +{{- range $key := list "history.enabled" "history.readOnly" "flows.enabled" }} +{{- $v := dig (index (splitList "." $key) 0) (index (splitList "." $key) 1) false $e -}} +{{- if not (kindIs "bool" $v) }}{{ fail (printf "eksIngress.%s must be true or false" $key) }}{{ end -}} +{{- $_ := set $bool $key $v -}} +{{- end -}} +{{- $name := dig "name" "agent-relay" $e | toString -}} +{{- if not (include "agent-relay.isDnsName" (dict "name" $name "max" 253)) }}{{ fail "eksIngress.name must be a DNS subdomain name" }}{{ end -}} +{{- $class := dig "className" "alb" $e | toString -}} +{{- if not (include "agent-relay.isDnsName" (dict "name" $class "max" 253)) }}{{ fail "eksIngress.className must be an IngressClass name" }}{{ end -}} +{{- $domain := dig "domain" "" $e | toString -}} +{{- if not $domain }}{{ fail "eksIngress.domain (the base domain) is required" }}{{ end -}} +{{- /* history. must stay within 253 characters. */ -}} +{{- if not (include "agent-relay.isDnsName" (dict "name" $domain "max" 245)) }}{{ fail "eksIngress.domain must be a lowercase DNS name (the base domain, no scheme or path): dot-separated labels of 1-63 letters, digits and inner hyphens, at most 245 characters" }}{{ end -}} +{{- $cert := dig "certificateArn" "" $e | toString -}} +{{- if not $cert }}{{ fail "eksIngress.certificateArn (the ACM certificate ARN) is required" }}{{ end -}} +{{- if not (regexMatch "^arn:[^\\s'\"]+$" $cert) }}{{ fail "eksIngress.certificateArn must be an ACM certificate ARN (arn:...)" }}{{ end -}} +{{- $history := index $bool "history.enabled" -}} +{{- $readOnly := index $bool "history.readOnly" -}} +{{- $flows := index $bool "flows.enabled" -}} +{{- /* Unset (null) ingest follows the script: on whenever the history host is +published and not read-only. */ -}} +{{- $ingest := index (dig "ingest" dict $e) "enabled" -}} +{{- if kindIs "invalid" $ingest }}{{ $ingest = and $history (not $readOnly) }}{{ end -}} +{{- if not (kindIs "bool" $ingest) }}{{ fail "eksIngress.ingest.enabled must be true, false or null" }}{{ end -}} +{{- $ingestSvc := dig "ingest" "serviceName" "agent-relay-ingest-edge" $e | toString -}} +{{- $ingestPort := dig "ingest" "servicePort" 80 $e | toString -}} +{{- $readSvc := dig "readApi" "serviceName" "agent-relay-read-api" $e | toString -}} +{{- $readPort := dig "readApi" "servicePort" 80 $e | toString -}} +{{- $flowsSvc := dig "flows" "serviceName" "relayfile-control" $e | toString -}} +{{- $flowsPort := dig "flows" "servicePort" 3100 $e | toString -}} +{{- $chiefSvc := dig "chief" "serviceName" "" $e | toString -}} +{{- $chiefPort := dig "chief" "servicePort" 3200 $e | toString -}} +{{- $controlSvc := dig "control" "serviceName" "relay-control" $e | toString -}} +{{- $webappSvc := dig "webapp" "serviceName" "relay-webapp" $e | toString -}} +{{- $validPort := dict -}} +{{- range $p := list $ingestPort $readPort $flowsPort $chiefPort }} +{{- $_ := set $validPort $p (and (regexMatch $port $p) (le (atoi $p) 65535)) -}} +{{- end -}} +{{- if $chiefSvc }} +{{- if not (and (le (len $chiefSvc) 63) (regexMatch $label $chiefSvc)) }}{{ fail "eksIngress.chief.serviceName must be a DNS label" }}{{ end -}} +{{- if not (index $validPort $chiefPort) }}{{ fail "eksIngress.chief.servicePort must be 1-65535" }}{{ end -}} +{{- end -}} +{{- range $svc := list $controlSvc $webappSvc }} +{{- if not (and (le (len $svc) 63) (regexMatch $label $svc)) }}{{ fail "eksIngress.control.serviceName and eksIngress.webapp.serviceName must be DNS labels" }}{{ end -}} +{{- end -}} +{{- if and $history (not $readOnly) (not $ingest) }}{{ fail "eksIngress.ingest.enabled=false needs eksIngress.history.readOnly=true: the history host has no standalone RelayHistory backend" }}{{ end -}} +{{- if $flows }} +{{- if not (and (le (len $flowsSvc) 63) (regexMatch $label $flowsSvc)) }}{{ fail "eksIngress.flows.serviceName must be a DNS label" }}{{ end -}} +{{- if not (index $validPort $flowsPort) }}{{ fail "eksIngress.flows.servicePort must be 1-65535" }}{{ end -}} +{{- end -}} +{{- if $readOnly }} +{{- if not (and $history (not $ingest)) }}{{ fail "eksIngress.history.readOnly=true needs eksIngress.history.enabled=true and eksIngress.ingest.enabled=false" }}{{ end -}} +{{- if not (and (le (len $readSvc) 63) (regexMatch $label $readSvc)) }}{{ fail "eksIngress.readApi.serviceName must be a DNS label" }}{{ end -}} +{{- if not (index $validPort $readPort) }}{{ fail "eksIngress.readApi.servicePort must be 1-65535" }}{{ end -}} +{{- end -}} +{{- if $ingest }} +{{- if not $history }}{{ fail "eksIngress.ingest.enabled=true needs the history host (eksIngress.history.enabled=true)" }}{{ end -}} +{{- range $svc := list $ingestSvc $readSvc }} +{{- if not (and (le (len $svc) 63) (regexMatch $label $svc)) }}{{ fail "eksIngress.ingest.serviceName and eksIngress.readApi.serviceName must be DNS labels" }}{{ end -}} +{{- end -}} +{{- range $p := list $ingestPort $readPort }} +{{- if not (index $validPort $p) }}{{ fail "eksIngress.ingest.servicePort and eksIngress.readApi.servicePort must be 1-65535" }}{{ end -}} +{{- end -}} +{{- end -}} +{{- /* Dashboard-host paths sent to the control API. The default is the list +the script emits; a path here is a Prefix route (Exact plus "/*" with +Chief on), so it may not be "/", end in "/", or hold "*" or "//". */ -}} +{{- $controlPaths := dig "controlPaths" (list "/cloud/api" "/login" "/relay-node") $e -}} +{{- if not (kindIs "slice" $controlPaths) }}{{ fail "eksIngress.controlPaths must be a list of paths" }}{{ end -}} +{{- range $p := $controlPaths }} +{{- if not (and (kindIs "string" $p) (regexMatch "^(/[-A-Za-z0-9._~%!$&'()+,;=:@]+)+$" $p)) }}{{ fail (printf "eksIngress.controlPaths entry %v must be an absolute path such as /cloud/api: not \"/\", no trailing \"/\", \"//\", \"*\" or spaces" $p) }}{{ end -}} +{{- end -}} +{{- if ne (len $controlPaths) (len (uniq $controlPaths)) }}{{ fail "eksIngress.controlPaths must not repeat a path" }}{{ end -}} +{{- $flowsRoutes := list -}} +{{- if $flows }}{{ range $r := list "flows" "workers" "workflows" }}{{ $flowsRoutes = append $flowsRoutes $r }}{{ end }}{{ end -}} +{{- $deny := "backend: { service: { name: edge-deny, port: { name: use-annotation } } }" -}} +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: {{ $name }} + namespace: {{ .Release.Namespace | quote }} + labels: + app.kubernetes.io/name: agent-relay + app.kubernetes.io/instance: {{ .Release.Name }} + annotations: + {{- if or $readOnly $ingest }} + alb.ingress.kubernetes.io/healthcheck-path: /healthz + {{- end }} + alb.ingress.kubernetes.io/scheme: internet-facing + alb.ingress.kubernetes.io/target-type: ip + alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]' + alb.ingress.kubernetes.io/ssl-redirect: "443" + alb.ingress.kubernetes.io/certificate-arn: {{ $cert | quote }} + alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-TLS13-1-2-2021-06 + alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=3600 + alb.ingress.kubernetes.io/actions.edge-deny: >- + {"type":"fixed-response","fixedResponseConfig":{"contentType":"text/plain","statusCode":"404","messageBody":"not found"}} +spec: + ingressClassName: {{ $class }} + tls: + - hosts: + - {{ $domain }} + - api.{{ $domain }} + - cast.{{ $domain }} + {{- if $history }} + - history.{{ $domain }} + {{- end }} + rules: + - host: {{ $domain }} + http: + paths: + {{- range $r := $flowsRoutes }} + - path: /cloud/api/v1/{{ $r }} + pathType: Prefix + backend: { service: { name: {{ $flowsSvc }}, port: { number: {{ $flowsPort }} } } } + {{- end }} + {{- if $chiefSvc }} + {{- /* Chief routes need a wildcard for the workspace id; wildcards are + ImplementationSpecific and the controller orders Exact, Prefix, then + ImplementationSpecific, so this host's Prefix paths become Exact plus + ordered wildcards. */}} + {{- range $p := $controlPaths }} + - path: {{ $p | quote }} + pathType: Exact + backend: { service: { name: {{ $controlSvc }}, port: { number: 80 } } } + {{- end }} + {{- range $prefix := list "/cloud/api/v1" "/api/v1" }} + {{- range $suffix := list "chiefs" "chiefs/*" }} + - path: "{{ $prefix }}/workspaces/*/{{ $suffix }}" + pathType: ImplementationSpecific + backend: { service: { name: {{ $chiefSvc }}, port: { number: {{ $chiefPort }} } } } + {{- end }} + {{- end }} + {{- range $p := $controlPaths }} + - path: "{{ $p }}/*" + pathType: ImplementationSpecific + backend: { service: { name: {{ $controlSvc }}, port: { number: 80 } } } + {{- end }} + - path: "/*" + pathType: ImplementationSpecific + backend: { service: { name: {{ $webappSvc }}, port: { number: 8080 } } } + {{- else }} + {{- range $p := $controlPaths }} + - path: {{ $p | quote }} + pathType: Prefix + backend: { service: { name: {{ $controlSvc }}, port: { number: 80 } } } + {{- end }} + - path: / + pathType: Prefix + backend: { service: { name: {{ $webappSvc }}, port: { number: 8080 } } } + {{- end }} + - host: api.{{ $domain }} + http: + paths: + - path: /internal + pathType: Prefix + {{ $deny }} + - path: "//internal*" + pathType: ImplementationSpecific + {{ $deny }} + {{- range $base := list "/cloud/api/v1" "/api/v1" }} + {{- range $r := $flowsRoutes }} + - path: {{ $base }}/{{ $r }} + pathType: Prefix + backend: { service: { name: {{ $flowsSvc }}, port: { number: {{ $flowsPort }} } } } + {{- end }} + {{- end }} + {{- if $chiefSvc }} + {{- range $prefix := list "/cloud/api/v1" "/api/v1" }} + {{- range $suffix := list "chiefs" "chiefs/*" }} + - path: "{{ $prefix }}/workspaces/*/{{ $suffix }}" + pathType: ImplementationSpecific + backend: { service: { name: {{ $chiefSvc }}, port: { number: {{ $chiefPort }} } } } + {{- end }} + {{- end }} + {{- end }} + - path: "/*" + pathType: ImplementationSpecific + backend: { service: { name: {{ $controlSvc }}, port: { number: 80 } } } + - host: cast.{{ $domain }} + http: + paths: + - path: /internal + pathType: Prefix + {{ $deny }} + - path: "//internal*" + pathType: ImplementationSpecific + {{ $deny }} + - path: /v1/workspaces + pathType: Exact + {{ $deny }} + - path: /v1/workspaces/ + pathType: Exact + {{ $deny }} + - path: "/*" + pathType: ImplementationSpecific + backend: { service: { name: relaycast, port: { number: 8787 } } } + {{- if $history }} + - host: history.{{ $domain }} + http: + paths: + {{- if not $readOnly }} + {{- range $p := list "/v1/delivery/batches" "/v1/turns/batches" "/v1/delivery/limits" "/v1/auth/token/refresh" }} + - path: {{ $p }} + pathType: Exact + backend: { service: { name: {{ $ingestSvc }}, port: { number: {{ $ingestPort }} } } } + {{- end }} + {{- end }} + {{- range $p := list "/v1/delivery/records" "/v1/cli/login" "/v1/events" "/v1/auth/token/revoke" }} + - path: {{ $p }} + pathType: Exact + backend: { service: { name: {{ $readSvc }}, port: { number: {{ $readPort }} } } } + {{- end }} + {{- range $p := list "/v1/sessions" "/v1/digest" }} + - path: {{ $p }} + pathType: Prefix + backend: { service: { name: {{ $readSvc }}, port: { number: {{ $readPort }} } } } + {{- end }} + {{- if not $readOnly }} + {{- /* The ingest edge owns /v1/delivery/*, /v1/turns/* and + /v1/onboarding/*, except delivery records (Exact, above). */}} + {{- range $p := list "/v1/delivery" "/v1/turns" "/v1/onboarding" }} + - path: {{ $p }} + pathType: Prefix + backend: { service: { name: {{ $ingestSvc }}, port: { number: {{ $ingestPort }} } } } + {{- end }} + {{- end }} + - path: /internal + pathType: Prefix + {{ $deny }} + - path: "//internal*" + pathType: ImplementationSpecific + {{ $deny }} + {{- /* Everything not named above is a fixed 404 at the edge. */}} + - path: "/*" + pathType: ImplementationSpecific + {{ $deny }} + {{- end }} +{{- end -}} diff --git a/charts/agent-relay/values-eks.yaml b/charts/agent-relay/values-eks.yaml index f398a608..abfec685 100644 --- a/charts/agent-relay/values-eks.yaml +++ b/charts/agent-relay/values-eks.yaml @@ -10,12 +10,15 @@ # backend, the Sessions read API and the ingest edge on one history origin; no # RelayAuth. Enable sessions-pipeline.sessionReadApi and .ingestEdge in the # site values (docs/self-hosted-native-ingest.md), and publish the ALB with -# scripts/selfhost/eks-alb-ingress.sh and EKS_ALB_INGEST=true. +# scripts/selfhost/eks-alb-ingress.sh and EKS_ALB_INGEST=true (the documented +# way), or render the same Ingress from this release with eksIngress below +# (README: EKS ALB Ingress; adopting a script-applied Ingress is described there). # values-eks-native.yaml is now an empty compatibility overlay. # # AWS mapping of the GCP profile: # GKE Ingress + ManagedCertificate -> one ALB Ingress with host rules and an -# ACM certificate (scripts/selfhost/eks-alb-ingress.sh); +# ACM certificate (scripts/selfhost/eks-alb-ingress.sh, +# or eksIngress.enabled=true); # EKS Auto Mode's built-in ALB controller # or the AWS Load Balancer Controller # GKE Workload Identity -> EKS Pod Identity associations @@ -50,6 +53,17 @@ global: gkeIngress: enabled: false +# Off: the edge is applied with scripts/selfhost/eks-alb-ingress.sh. To render +# it from the release instead, set in the site values (options and defaults in +# values.yaml mirror the script's variables): +# eksIngress: +# enabled: true +# domain: relay.example.com # REQUIRED OVERRIDE +# certificateArn: arn:aws:acm:... # REQUIRED OVERRIDE (covers all four names) +# controlPaths: [/cloud/api, /login, /relay-node] # pin the live ALB's list +eksIngress: + enabled: false + relaycast: fullnameOverride: relaycast image: diff --git a/charts/agent-relay/values.yaml b/charts/agent-relay/values.yaml index 1188bdac..b2676735 100644 --- a/charts/agent-relay/values.yaml +++ b/charts/agent-relay/values.yaml @@ -196,6 +196,56 @@ gkeIngress: # for the actors host; durable-actors sockets are long-lived. actorsBackendTimeoutSec: 3600 +# EKS edge (templates/eks-alb-ingress.yaml): one internet-facing ALB Ingress +# with host rules, exactly what scripts/selfhost/eks-alb-ingress.sh prints for +# the same options (each option below names the script variable it mirrors; +# defaults are the script's). The script stays the documented way; this is the +# alternative that makes the edge part of the release. Exclusive with +# gkeIngress. Backends are named, not looked up: Flows and Chief are separate +# releases, and a read-only history host may name a read API Service this +# release does not install. README: EKS ALB Ingress. +eksIngress: + enabled: false + name: agent-relay # EKS_ALB_INGRESS_NAME + className: alb # EKS_INGRESS_CLASS + # Required when enabled: the base domain (, api., cast., history.) + # and the ACM certificate that covers all four names. + domain: "" # EKS_ALB_DOMAIN + certificateArn: "" # EKS_ALB_CERT_ARN + history: + enabled: true # EKS_ALB_HISTORY + # Read API only on history. (needs ingest.enabled=false). + readOnly: false # EKS_ALB_HISTORY_READ_ONLY + ingest: + # null: on whenever history.enabled and not history.readOnly. + enabled: null # EKS_ALB_INGEST + serviceName: agent-relay-ingest-edge # EKS_ALB_INGEST_SERVICE + servicePort: 80 # EKS_ALB_INGEST_PORT + readApi: + serviceName: agent-relay-read-api # EKS_ALB_READ_API_SERVICE + servicePort: 80 # EKS_ALB_READ_API_PORT + # Flows on relayfile-control (a separate release). + flows: + enabled: false # EKS_ALB_RELAYFILE_CONTROL + serviceName: relayfile-control # EKS_ALB_RELAYFILE_CONTROL_SERVICE + servicePort: 3100 # EKS_ALB_RELAYFILE_CONTROL_PORT + # Chief on a separately installed relaychief-control; empty serviceName = off. + chief: + serviceName: "" # EKS_ALB_CHIEF_SERVICE + servicePort: 3200 # EKS_ALB_CHIEF_PORT + control: + serviceName: relay-control # EKS_ALB_CONTROL_SERVICE + webapp: + serviceName: relay-webapp # EKS_ALB_WEBAPP_SERVICE + # Dashboard-host () paths sent to the control API, in order, ahead + # of the dashboard catch-all. The script has no option for this; the default + # is the list it emits. Pin the list your live ALB has before an upgrade + # that changes the default. + controlPaths: + - /cloud/api + - /login + - /relay-node + # Child values are intentionally not populated with credentials or image # placeholders here. Every leaf chart fails closed until its existingSecret, # public origins and immutable image digest are supplied. diff --git a/ci/agent-relay/test_eks_alb_ingress.py b/ci/agent-relay/test_eks_alb_ingress.py new file mode 100644 index 00000000..e959b90f --- /dev/null +++ b/ci/agent-relay/test_eks_alb_ingress.py @@ -0,0 +1,258 @@ +#!/usr/bin/env python3 +"""The chart's eksIngress renders the Ingress scripts/selfhost/eks-alb-ingress.sh prints. + +Every case runs the real script and a real `helm template` with the same +options and compares the parsed objects: annotations and spec deep-equal +(rule and path order included), name and namespace equal. Helm's own labels +are the only allowed difference. Refusals fail in both. Needs the chart's +dependencies built (ci/agent-relay/render.sh does it first in CI). +""" +import json +import os +from pathlib import Path +import subprocess +import unittest + +import yaml + +ROOT = Path(__file__).resolve().parents[2] +CHART = 'charts/agent-relay' +NAMESPACE = 'relay-eks' +BASE_ENV = {'EKS_ALB_DOMAIN': 'relay.example.com', + 'EKS_ALB_CERT_ARN': 'arn:aws:acm:us-east-1:111111111111:certificate/00000000-0000-0000-0000-000000000000'} + +# Script variable -> eksIngress option path. +OPTIONS = { + 'EKS_ALB_INGRESS_NAME': 'name', + 'EKS_INGRESS_CLASS': 'className', + 'EKS_ALB_DOMAIN': 'domain', + 'EKS_ALB_CERT_ARN': 'certificateArn', + 'EKS_ALB_HISTORY': 'history.enabled', + 'EKS_ALB_HISTORY_READ_ONLY': 'history.readOnly', + 'EKS_ALB_INGEST': 'ingest.enabled', + 'EKS_ALB_INGEST_SERVICE': 'ingest.serviceName', + 'EKS_ALB_INGEST_PORT': 'ingest.servicePort', + 'EKS_ALB_READ_API_SERVICE': 'readApi.serviceName', + 'EKS_ALB_READ_API_PORT': 'readApi.servicePort', + 'EKS_ALB_RELAYFILE_CONTROL': 'flows.enabled', + 'EKS_ALB_RELAYFILE_CONTROL_SERVICE': 'flows.serviceName', + 'EKS_ALB_RELAYFILE_CONTROL_PORT': 'flows.servicePort', + 'EKS_ALB_CHIEF_SERVICE': 'chief.serviceName', + 'EKS_ALB_CHIEF_PORT': 'chief.servicePort', + 'EKS_ALB_CONTROL_SERVICE': 'control.serviceName', + 'EKS_ALB_WEBAPP_SERVICE': 'webapp.serviceName', +} + +CASES = { + 'defaults': {}, + 'history off': {'EKS_ALB_HISTORY': 'false'}, + 'history read-only': {'EKS_ALB_HISTORY_READ_ONLY': 'true'}, + 'read-only, ingest explicitly off, own read API': { + 'EKS_ALB_HISTORY_READ_ONLY': 'true', 'EKS_ALB_INGEST': 'false', + 'EKS_ALB_READ_API_SERVICE': 'other-read-api', 'EKS_ALB_READ_API_PORT': '8080'}, + 'ingest off (history off)': {'EKS_ALB_HISTORY': 'false', 'EKS_ALB_INGEST': 'false'}, + 'ingest explicitly on': {'EKS_ALB_INGEST': 'true'}, + 'flows on': {'EKS_ALB_RELAYFILE_CONTROL': 'true'}, + 'chief on': {'EKS_ALB_CHIEF_SERVICE': 'relaychief-control'}, + 'flows + chief + read-only': {'EKS_ALB_RELAYFILE_CONTROL': 'true', + 'EKS_ALB_CHIEF_SERVICE': 'relaychief-control', + 'EKS_ALB_HISTORY_READ_ONLY': 'true'}, + 'overridden names, ports, ingress name and class': { + 'EKS_ALB_INGRESS_NAME': 'relay-edge', 'EKS_INGRESS_CLASS': 'alb-internal', + 'EKS_ALB_DOMAIN': 'edge.example.com', + 'EKS_ALB_INGEST_SERVICE': 'ingest', 'EKS_ALB_INGEST_PORT': '8081', + 'EKS_ALB_READ_API_SERVICE': 'reader', 'EKS_ALB_READ_API_PORT': '8082', + 'EKS_ALB_RELAYFILE_CONTROL': 'true', 'EKS_ALB_RELAYFILE_CONTROL_SERVICE': 'flows-api', + 'EKS_ALB_RELAYFILE_CONTROL_PORT': '9100', + 'EKS_ALB_CHIEF_SERVICE': 'chief-api', 'EKS_ALB_CHIEF_PORT': '9200', + 'EKS_ALB_CONTROL_SERVICE': 'relay-cloud-api', 'EKS_ALB_WEBAPP_SERVICE': 'relay-dashboard'}, +} + +# (script env, chart-only extra values, message fragment in the chart error). +REFUSALS = { + 'domain missing': ({'EKS_ALB_DOMAIN': ''}, {}, 'eksIngress.domain'), + 'certificate missing': ({'EKS_ALB_CERT_ARN': ''}, {}, 'eksIngress.certificateArn'), + 'read-only with ingest on': ({'EKS_ALB_HISTORY_READ_ONLY': 'true', 'EKS_ALB_INGEST': 'true'}, {}, 'history.readOnly=true needs'), + 'read-only with history off': ({'EKS_ALB_HISTORY_READ_ONLY': 'true', 'EKS_ALB_HISTORY': 'false'}, {}, 'history.readOnly=true needs'), + 'ingest off without read-only': ({'EKS_ALB_INGEST': 'false'}, {}, 'needs eksIngress.history.readOnly=true'), + 'ingest on with history off': ({'EKS_ALB_HISTORY': 'false', 'EKS_ALB_INGEST': 'true'}, {}, 'needs the history host'), + 'control Service not a label': ({'EKS_ALB_CONTROL_SERVICE': 'Relay_Control'}, {}, 'must be DNS labels'), + 'webapp Service not a label': ({'EKS_ALB_WEBAPP_SERVICE': 'a' * 64}, {}, 'must be DNS labels'), + 'ingest Service not a label': ({'EKS_ALB_INGEST_SERVICE': 'ingest.edge'}, {}, 'must be DNS labels'), + 'ingest port 0': ({'EKS_ALB_INGEST_PORT': '0'}, {}, 'must be 1-65535'), + 'read API port 65536': ({'EKS_ALB_READ_API_PORT': '65536'}, {}, 'must be 1-65535'), + 'read-only read API Service not a label': ({'EKS_ALB_HISTORY_READ_ONLY': 'true', 'EKS_ALB_READ_API_SERVICE': '-reader'}, {}, 'readApi.serviceName must be a DNS label'), + 'flows Service not a label': ({'EKS_ALB_RELAYFILE_CONTROL': 'true', 'EKS_ALB_RELAYFILE_CONTROL_SERVICE': 'Flows'}, {}, 'flows.serviceName must be a DNS label'), + 'flows port 70000': ({'EKS_ALB_RELAYFILE_CONTROL': 'true', 'EKS_ALB_RELAYFILE_CONTROL_PORT': '70000'}, {}, 'flows.servicePort must be 1-65535'), + 'chief Service not a label': ({'EKS_ALB_CHIEF_SERVICE': 'chief_control'}, {}, 'chief.serviceName must be a DNS label'), + 'chief port 0': ({'EKS_ALB_CHIEF_SERVICE': 'relaychief-control', 'EKS_ALB_CHIEF_PORT': '0'}, {}, 'chief.servicePort must be 1-65535'), +} + + +def script(env, namespace=NAMESPACE): + full = {k: v for k, v in os.environ.items() if not k.startswith(('EKS_ALB_', 'EKS_INGRESS_'))} + full.update(BASE_ENV) + full.update(env) + return subprocess.run(['bash', 'scripts/selfhost/eks-alb-ingress.sh', namespace], cwd=ROOT, + env=full, capture_output=True, text=True) + + +def chart_values(env): + values = {'enabled': True} + for key, value in {**BASE_ENV, **env}.items(): + node = values + *parents, leaf = OPTIONS[key].split('.') + for part in parents: + node = node.setdefault(part, {}) + if value in ('true', 'false'): + value = value == 'true' + elif leaf == 'servicePort' and value.isdigit(): + value = int(value) + node[leaf] = value + return values + + +def helm(values, namespace=NAMESPACE, extra=()): + command = ['helm', 'template', 'proof', CHART, '-n', namespace, '--no-hooks', + '-f', f'{CHART}/values-eks.yaml', '-f', f'{CHART}/values-eks-rds-native.yaml', + '--set-json', 'eksIngress=' + json.dumps(values), + '--show-only', 'templates/eks-alb-ingress.yaml', *extra] + return subprocess.run(command, cwd=ROOT, capture_output=True, text=True) + + +def ingress(stdout): + docs = [d for d in yaml.safe_load_all(stdout) if d] + assert len(docs) == 1 and docs[0]['kind'] == 'Ingress', docs + return docs[0] + + +def comparable(doc): + meta = doc['metadata'] + return {'apiVersion': doc['apiVersion'], 'kind': doc['kind'], 'name': meta['name'], + 'namespace': meta['namespace'], 'annotations': meta.get('annotations', {}), 'spec': doc['spec']} + + +def paths(doc, host): + rule = next(r for r in doc['spec']['rules'] if r['host'] == host) + return [(p['path'], p['pathType']) for p in rule['http']['paths']] + + +class EksAlbIngress(unittest.TestCase): + def assert_same(self, env, chart_extra=None): + want = script(env) + self.assertEqual(want.returncode, 0, want.stderr) + values = chart_values(env) + values.update(chart_extra or {}) + got = helm(values) + self.assertEqual(got.returncode, 0, got.stderr) + g, w = ingress(got.stdout), ingress(want.stdout) + self.assertEqual(comparable(g), comparable(w)) + return g, w + + def test_matrix_equals_script(self): + for name, env in CASES.items(): + with self.subTest(case=name): + self.assert_same(env) + + def test_off_by_default(self): + result = subprocess.run(['helm', 'template', 'proof', CHART, '-n', NAMESPACE, '--no-hooks', + '-f', f'{CHART}/values-eks.yaml', '-f', f'{CHART}/values-eks-rds-native.yaml'], + cwd=ROOT, capture_output=True, text=True) + self.assertEqual(result.returncode, 0, result.stderr) + docs = [d for d in yaml.safe_load_all(result.stdout) if d] + self.assertFalse([d for d in docs if d['kind'] == 'Ingress' and d['spec'].get('ingressClassName') == 'alb']) + self.assertNotIn('templates/eks-alb-ingress.yaml', result.stdout) + + def test_enabled_must_be_a_boolean(self): + # A string is truthy in a template: "false" would otherwise render the edge. + for value in ('false', 'true', 'no'): + with self.subTest(enabled=value): + result = helm(chart_values({}), extra=['--set-string', f'eksIngress.enabled={value}']) + self.assertNotEqual(result.returncode, 0, f'chart accepted enabled={value!r}') + self.assertIn('eksIngress.enabled must be true or false', result.stderr) + self.assertNotIn('kind: Ingress', result.stdout) + + def test_chart_only_labels_differ(self): + got = ingress(helm(chart_values({})).stdout) + self.assertEqual(got['metadata']['labels'], + {'app.kubernetes.io/name': 'agent-relay', 'app.kubernetes.io/instance': 'proof'}) + + def test_namespace_is_the_release_namespace(self): + want = ingress(script({}, namespace='other-ns').stdout) + got = ingress(helm(chart_values({}), namespace='other-ns').stdout) + self.assertEqual(got['metadata']['namespace'], 'other-ns') + self.assertEqual(comparable(got), comparable(want)) + + def test_backends_need_no_component_of_this_release(self): + # Flows, Chief and a foreign read API on a release that installs none of + # them (and with the sessions pipeline's own Services absent). + env = CASES['flows + chief + read-only'] + values = chart_values(env) + result = helm(values, extra=['--set', 'components.sessionsPipeline=false']) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(comparable(ingress(result.stdout)), comparable(ingress(script(env).stdout))) + + def test_control_paths_default_is_the_script_list(self): + for env in ({}, {'EKS_ALB_CHIEF_SERVICE': 'relaychief-control'}): + with self.subTest(env=env): + self.assert_same(env, {'controlPaths': ['/cloud/api', '/login', '/relay-node']}) + + def test_control_paths_drop_only_the_relay_node_rules(self): + for env in ({}, {'EKS_ALB_CHIEF_SERVICE': 'relaychief-control'}, {'EKS_ALB_RELAYFILE_CONTROL': 'true'}): + with self.subTest(env=env): + want = ingress(script(env).stdout) + result = helm({**chart_values(env), 'controlPaths': ['/cloud/api', '/login']}) + self.assertEqual(result.returncode, 0, result.stderr) + got = ingress(result.stdout) + for rule in want['spec']['rules']: + if rule['host'] == 'relay.example.com': + rule['http']['paths'] = [p for p in rule['http']['paths'] + if not p['path'].startswith('/relay-node')] + self.assertEqual(comparable(got), comparable(want)) + self.assertFalse([p for p in paths(got, 'relay.example.com') if 'relay-node' in p[0]]) + + def test_control_paths_extend_in_order(self): + result = helm({**chart_values({}), 'controlPaths': ['/cloud/api', '/login', '/relay-node', '/mcp', '/.well-known/openid-configuration']}) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(paths(ingress(result.stdout), 'relay.example.com'), + [('/cloud/api', 'Prefix'), ('/login', 'Prefix'), ('/relay-node', 'Prefix'), ('/mcp', 'Prefix'), + ('/.well-known/openid-configuration', 'Prefix'), ('/', 'Prefix')]) + + def test_refusals_match_the_script(self): + for name, (env, extra, message) in REFUSALS.items(): + with self.subTest(case=name): + self.assertNotEqual(script(env).returncode, 0, f'script accepted {name}') + values = {**chart_values(env), **extra} + result = helm(values) + self.assertNotEqual(result.returncode, 0, f'chart accepted {name}') + self.assertIn(message, result.stderr) + + def test_chart_only_refusals(self): + bad = { + 'both edges': ({}, ['--set', 'gkeIngress.enabled=true'], 'exclusive'), + 'string boolean': ({'history': {'enabled': 'yes'}}, [], 'history.enabled must be true or false'), + 'string ingest': ({'ingest': {'enabled': 'true'}}, [], 'ingest.enabled must be true, false or null'), + 'control path /': ({'controlPaths': ['/']}, [], 'controlPaths entry'), + 'control path wildcard': ({'controlPaths': ['/cloud/*']}, [], 'controlPaths entry'), + 'control path trailing slash': ({'controlPaths': ['/login/']}, [], 'controlPaths entry'), + 'control path relative': ({'controlPaths': ['login']}, [], 'controlPaths entry'), + 'control path repeated': ({'controlPaths': ['/login', '/login']}, [], 'must not repeat'), + 'control paths not a list': ({'controlPaths': '/login'}, [], 'must be a list'), + 'domain with scheme': ({'domain': 'https://relay.example.com'}, [], 'lowercase DNS name'), + 'domain with an empty label': ({'domain': 'relay..example.com'}, [], 'lowercase DNS name'), + 'domain label starting with a hyphen': ({'domain': 'relay.-example.com'}, [], 'lowercase DNS name'), + 'domain label ending with a hyphen': ({'domain': 'relay-.example.com'}, [], 'lowercase DNS name'), + 'domain label over 63': ({'domain': 'a' * 64 + '.example.com'}, [], 'lowercase DNS name'), + 'domain over 245': ({'domain': '.'.join(['a' * 60] * 4) + '.com'}, [], 'lowercase DNS name'), + 'name with an empty label': ({'name': 'agent..relay'}, [], 'eksIngress.name must be a DNS subdomain name'), + 'class label starting with a hyphen': ({'className': 'alb.-x'}, [], 'eksIngress.className must be an IngressClass name'), + } + for name, (values, extra, message) in bad.items(): + with self.subTest(case=name): + result = helm({**chart_values({}), **values}, extra=extra) + self.assertNotEqual(result.returncode, 0, f'chart accepted {name}') + self.assertIn(message, result.stderr) + + +if __name__ == '__main__': + unittest.main() diff --git a/docs/self-hosted-eks.md b/docs/self-hosted-eks.md index cbde940a..0d1d13c1 100644 --- a/docs/self-hosted-eks.md +++ b/docs/self-hosted-eks.md @@ -34,7 +34,7 @@ Files: | File | What it is | | --- | --- | | [`charts/agent-relay/values-eks.yaml`](../charts/agent-relay/values-eks.yaml) | Customer-neutral EKS values. Replace every `REQUIRED OVERRIDE`. | -| [`scripts/selfhost/eks-alb-ingress.sh`](../scripts/selfhost/eks-alb-ingress.sh) | Prints the ALB Ingress (one ALB, four hosts, ACM TLS, edge denials). The chart ships only the GKE edge. | +| [`scripts/selfhost/eks-alb-ingress.sh`](../scripts/selfhost/eks-alb-ingress.sh) | Prints the ALB Ingress (one ALB, four hosts, ACM TLS, edge denials). The documented way; the chart's opt-in `eksIngress` renders the same Ingress from the release ([EKS ALB Ingress](../charts/agent-relay/README.md#eks-alb-ingress)). | | [`scripts/selfhost/eks-proof-dns.sh`](../scripts/selfhost/eks-proof-dns.sh) | ACM certificate (request, wait, delete) and the DNS records (validation CNAMEs, host CNAMEs to the ALB) in the parent Cloudflare zone, removed by recorded id. | | [`scripts/selfhost/eks-proof-sandbox.sh`](../scripts/selfhost/eks-proof-sandbox.sh) | The proof substrate: VPC, EKS Auto Mode cluster, IAM roles, S3 bucket, Pod Identity, StorageClass/IngressClass. Pippen conventions. | | `scripts/selfhost/eks-proof-install.sh` (removed in agent-relay 0.7.0; see git history) | The 0.3.x/0.4.x proof installer: CNPG operator, the control-API and RelayHistory Postgres, generated Secret, Helm install, Ingress, installation org, HoopDb role bootstrap, uninstall. It only installed published charts that shipped the standalone RelayHistory, so it was removed with that component. | @@ -45,7 +45,7 @@ Files: | GKE (`values-gcp.yaml`) | EKS (`values-eks.yaml`) | | --- | --- | -| `gkeIngress` (GCE Ingress, ManagedCertificate, FrontendConfig, static IP) | `gkeIngress.enabled=false`; one ALB Ingress from `eks-alb-ingress.sh` (EKS Auto Mode's built-in ALB controller, IngressClass `alb`; the AWS Load Balancer Controller takes the same annotations): host rules for `` (dashboard, with `/cloud/api`, `/login` and `/relay-node` to the control API), `api.`, `cast.`, `history.`; HTTPS 443 with an ACM certificate for the four names, HTTP 80 redirects (301). Edge denials (`/internal`, `//internal*`, Relaycast workspace create) are ALB fixed-response 404 actions. | +| `gkeIngress` (GCE Ingress, ManagedCertificate, FrontendConfig, static IP) | `gkeIngress.enabled=false`; one ALB Ingress from `eks-alb-ingress.sh`, or the same Ingress from the chart with `eksIngress.enabled=true` (EKS Auto Mode's built-in ALB controller, IngressClass `alb`; the AWS Load Balancer Controller takes the same annotations): host rules for `` (dashboard, with `/cloud/api`, `/login` and `/relay-node` to the control API), `api.`, `cast.`, `history.`; HTTPS 443 with an ACM certificate for the four names, HTTP 80 redirects (301). Edge denials (`/internal`, `//internal*`, Relaycast workspace create) are ALB fixed-response 404 actions. | | ManagedCertificate | ACM certificate, DNS-validated | | NEG annotations, BackendConfig | `alb.ingress.kubernetes.io/healthcheck-path` on each Service; ALB idle timeout 3600 s for WebSocket/SSE | | GKE Workload Identity | The pinned 0.3.11 proof uses `-worker` and `-alleyoop` (one IAM role with S3 access to the bucket). New descriptive-name charts use `-session-ingest-api`; retain the actual legacy identity through a deliberate cutover. No IRSA/OIDC provider needed. | @@ -80,7 +80,9 @@ Steps (any DNS provider; with Route 53 create the same records there): 2. Create the validation CNAMEs (`eks-proof-dns.sh cf-validation`) and wait for `ISSUED` (`cert-wait`; about 2 minutes in the proof). 3. Install, then apply the Ingress with `EKS_ALB_DOMAIN` and - `EKS_ALB_CERT_ARN` (`eks-proof-install.sh install` does both). + `EKS_ALB_CERT_ARN` (`eks-proof-install.sh install` did both). Alternatively + set `eksIngress.enabled`, `eksIngress.domain` and + `eksIngress.certificateArn` and the release renders the Ingress. 4. Point the four names at the ALB's DNS name (CNAMEs, unproxied, or Route 53 alias records): `eks-proof-dns.sh alb`, then `cf-hosts`. @@ -218,9 +220,13 @@ adapt `ROLE_PATH`, `BOUNDARY` and `TAG_KEY` for your account. ## Gaps and follow-ups -1. **The chart has no EKS edge.** `gkeIngress` is GKE-only; the ALB Ingress - lives in `scripts/selfhost/eks-alb-ingress.sh`. An `albIngress` block in - the umbrella would make it one install. +1. **The EKS edge is opt-in in the chart.** `scripts/selfhost/eks-alb-ingress.sh` + stays the documented way. `eksIngress.enabled=true` renders the same + Ingress from the release (deep-equal to the script for the same options, + `ci/agent-relay/test_eks_alb_ingress.py`), plus `eksIngress.controlPaths` + to pin the dashboard-host control routes. An Ingress applied with the script + can be adopted by the release without a new ALB: see + [Adopting an Ingress that was applied with the script](../charts/agent-relay/README.md#adopting-an-ingress-that-was-applied-with-the-script). 2. **ALB rule order.** The controller orders a host's rules Exact, then Prefix, then ImplementationSpecific. `//internal*` must be an ImplementationSpecific path (Kubernetes refuses `//` in Exact/Prefix diff --git a/docs/self-hosted-requirements.md b/docs/self-hosted-requirements.md index 4a4a5c99..b02006de 100644 --- a/docs/self-hosted-requirements.md +++ b/docs/self-hosted-requirements.md @@ -170,7 +170,8 @@ Services image (sessions-pipeline-worker, alleyoop and the privileged SQL Jobs): | relayauth-0.1.2 | shipped | relayauth 0.1.2 | relayauth: `ghcr.io/agentworkforce/relayauth@sha256:c23c92d25fdf…` | | | License-annotation patch bump of 0.1.1. Superseded by 0.1.3. | | relayauth-0.1.1 | shipped | relayauth 0.1.1 | relayauth: `ghcr.io/agentworkforce/relayauth@sha256:c23c92d25fdf…` | | | relayauth 784582ef (machine-bound History assertion contract, relayauth #113). Superseded by 0.1.2. | | relayauth-0.1.0 | shipped | relayauth 0.1.0 | relayauth: `ghcr.io/agentworkforce/relayauth@sha256:eb71daa28e30…` | | | superseded by 0.1.1. | -| agent-relay-0.7.63 | unreleased | agent-relay 0.7.63
relay-control 0.4.22
relay-webapp 0.2.7
(deps: relaycast 0.3.16, relayauth 0.1.5, sessions-pipeline 0.8.52, relay-control 0.4.22, relay-webapp 0.2.7, relayflows 0.2.5, durable-actors 0.14.6-awf.4, relayfile 0.2.2, telemetry-collector 0.1.7, relay-chief 0.1.0, slack-relay 0.3.11, relaygate 0.1.10) | relaycast: `ghcr.io/agentworkforce/relaycast-self-host@sha256:ec062ef901af…`
relaycast-postgres: `ghcr.io/agentworkforce/relaycast-self-host@sha256:ec062ef901af…`
sessions-pipeline-worker: `ghcr.io/agentworkforce/sessions-pipeline@sha256:e51749eb2b7c…`
relay-control: `ghcr.io/agentworkforce/relay-control@sha256:d1b6ca95fab2…`
relay-webapp: `ghcr.io/agentworkforce/relay-webapp@sha256:bd295ccb7d13…` | | | Carries 0.7.62 and sessions-pipeline 0.8.52. The GCP, EKS and EKS RDS-native profiles move the services image to relayhistory-pipeline efdd4bb2 (ghcr.io/agentworkforce/sessions-pipeline@sha256:e51749eb, publish run 44489159041; issue 610 Phase 3, staged reconcile and streaming on by default; with streaming.enabled=false and stagedReconcile unset the staged path resolves off). It vendors relayscrubber 0.5.0-alpha.9 under Khaliqs recorded OVERRIDE of the relayscrubber-lead security block (customer-images.yaml); alpha.10 replaces it when it passes. Upgrade pre-steps for an existing database: the bulls96_legacy_export_privacy role (re-run the modes migration-roles bootstrap from the image) and the 058000 index prebuild or one allowBlockingIndexBuild. 28 migrations plus HoopDb 0005/0006 run in the migrate Job; the indexer restarts until 062110 (and 062140 with output facts) are applied. License FSL-1.1-ALv2. No upgrade rehearsal from a 7e9b2f0f database yet. | +| agent-relay-0.7.66 | unreleased | agent-relay 0.7.66
relay-control 0.4.22
relay-webapp 0.2.7
(deps: relaycast 0.3.16, relayauth 0.1.5, sessions-pipeline 0.8.52, relay-control 0.4.22, relay-webapp 0.2.7, relayflows 0.2.5, durable-actors 0.14.6-awf.4, relayfile 0.2.2, telemetry-collector 0.1.7, relay-chief 0.1.0, slack-relay 0.3.11, relaygate 0.1.10) | relaycast: `ghcr.io/agentworkforce/relaycast-self-host@sha256:ec062ef901af…`
relaycast-postgres: `ghcr.io/agentworkforce/relaycast-self-host@sha256:ec062ef901af…`
sessions-pipeline-worker: `ghcr.io/agentworkforce/sessions-pipeline@sha256:e51749eb2b7c…`
relay-control: `ghcr.io/agentworkforce/relay-control@sha256:d1b6ca95fab2…`
relay-webapp: `ghcr.io/agentworkforce/relay-webapp@sha256:bd295ccb7d13…` | | | Carries 0.7.63. Opt-in EKS ALB edge: eksIngress.enabled (default false) renders the internet-facing ALB Ingress that scripts/selfhost/eks-alb-ingress.sh prints, option for option, plus eksIngress.controlPaths (default /cloud/api, /login, /relay-node). Refuses eksIngress with gkeIngress. ci/agent-relay/test_eks_alb_ingress.py proves deep equality with the script over the option matrix. Default renders differ only in chart labels; images unchanged. Not applied to any cluster. | +| agent-relay-0.7.63 | **planned** | agent-relay 0.7.63
relay-control 0.4.22
relay-webapp 0.2.7
(deps: relaycast 0.3.16, relayauth 0.1.5, sessions-pipeline 0.8.52, relay-control 0.4.22, relay-webapp 0.2.7, relayflows 0.2.5, durable-actors 0.14.6-awf.4, relayfile 0.2.2, telemetry-collector 0.1.7, relay-chief 0.1.0, slack-relay 0.3.11, relaygate 0.1.10) | relaycast: `ghcr.io/agentworkforce/relaycast-self-host@sha256:ec062ef901af…`
relaycast-postgres: `ghcr.io/agentworkforce/relaycast-self-host@sha256:ec062ef901af…`
sessions-pipeline-worker: `ghcr.io/agentworkforce/sessions-pipeline@sha256:e51749eb2b7c…`
relay-control: `ghcr.io/agentworkforce/relay-control@sha256:d1b6ca95fab2…`
relay-webapp: `ghcr.io/agentworkforce/relay-webapp@sha256:bd295ccb7d13…` | | | Carries 0.7.62 and sessions-pipeline 0.8.52. The GCP, EKS and EKS RDS-native profiles move the services image to relayhistory-pipeline efdd4bb2 (ghcr.io/agentworkforce/sessions-pipeline@sha256:e51749eb, publish run 44489159041; issue 610 Phase 3, staged reconcile and streaming on by default; with streaming.enabled=false and stagedReconcile unset the staged path resolves off). It vendors relayscrubber 0.5.0-alpha.9 under Khaliqs recorded OVERRIDE of the relayscrubber-lead security block (customer-images.yaml); alpha.10 replaces it when it passes. Upgrade pre-steps for an existing database: the bulls96_legacy_export_privacy role (re-run the modes migration-roles bootstrap from the image) and the 058000 index prebuild or one allowBlockingIndexBuild. 28 migrations plus HoopDb 0005/0006 run in the migrate Job; the indexer restarts until 062110 (and 062140 with output facts) are applied. License FSL-1.1-ALv2. No upgrade rehearsal from a 7e9b2f0f database yet. | | agent-relay-0.7.62 | **planned** | agent-relay 0.7.62
relay-control 0.4.22
relay-webapp 0.2.7
(deps: relaycast 0.3.16, relayauth 0.1.5, sessions-pipeline 0.8.51, relay-control 0.4.22, relay-webapp 0.2.7, relayflows 0.2.5, durable-actors 0.14.6-awf.4, relayfile 0.2.2, telemetry-collector 0.1.7, relay-chief 0.1.0, slack-relay 0.3.11, relaygate 0.1.10) | relaycast: `ghcr.io/agentworkforce/relaycast-self-host@sha256:ec062ef901af…`
relaycast-postgres: `ghcr.io/agentworkforce/relaycast-self-host@sha256:ec062ef901af…`
sessions-pipeline-worker: `ghcr.io/agentworkforce/sessions-pipeline@sha256:77022c5b6a1d…`
relay-control: `ghcr.io/agentworkforce/relay-control@sha256:d1b6ca95fab2…`
relay-webapp: `ghcr.io/agentworkforce/relay-webapp@sha256:bd295ccb7d13…` | | | Carries 0.7.61 and sessions-pipeline 0.8.51. RDS migrate Jobs can select the name-agnostic separate-database preflight; fresh managed-database installs use the dedicated hoopdb_owner login and legacy upgrades can point migration/bootstrap Jobs at one short-lived Secret. | | agent-relay-0.7.61 | **planned** | agent-relay 0.7.61
relay-control 0.4.22
relay-webapp 0.2.7
(deps: relaycast 0.3.16, relayauth 0.1.5, sessions-pipeline 0.8.50, relay-control 0.4.22, relay-webapp 0.2.7, relayflows 0.2.5, durable-actors 0.14.6-awf.4, relayfile 0.2.2, telemetry-collector 0.1.7, relay-chief 0.1.0, slack-relay 0.3.11, relaygate 0.1.10) | relaycast: `ghcr.io/agentworkforce/relaycast-self-host@sha256:ec062ef901af…`
relaycast-postgres: `ghcr.io/agentworkforce/relaycast-self-host@sha256:ec062ef901af…`
sessions-pipeline-worker: `ghcr.io/agentworkforce/sessions-pipeline@sha256:77022c5b6a1d…`
relay-control: `ghcr.io/agentworkforce/relay-control@sha256:d1b6ca95fab2…`
relay-webapp: `ghcr.io/agentworkforce/relay-webapp@sha256:bd295ccb7d13…` | | | Carries 0.7.60 and sessions-pipeline 0.8.50 (searchIndexer.stagedReconcile.enabled, issue 610 Phase 3; no image pin changes). | | agent-relay-0.7.60 | **planned** | agent-relay 0.7.60
relay-control 0.4.22
relay-webapp 0.2.7
(deps: relaycast 0.3.16, relayauth 0.1.5, sessions-pipeline 0.8.49, relay-control 0.4.22, relay-webapp 0.2.7, relayflows 0.2.5, durable-actors 0.14.6-awf.4, relayfile 0.2.2, telemetry-collector 0.1.7, relay-chief 0.1.0, slack-relay 0.3.11, relaygate 0.1.10) | relaycast: `ghcr.io/agentworkforce/relaycast-self-host@sha256:ec062ef901af…`
relaycast-postgres: `ghcr.io/agentworkforce/relaycast-self-host@sha256:ec062ef901af…`
sessions-pipeline-worker: `ghcr.io/agentworkforce/sessions-pipeline@sha256:77022c5b6a1d…`
relay-control: `ghcr.io/agentworkforce/relay-control@sha256:d1b6ca95fab2…`
relay-webapp: `ghcr.io/agentworkforce/relay-webapp@sha256:bd295ccb7d13…` | | | Carries 0.7.59. Internal-only GKE edge, opt-in: gkeIngress.mode=internal (default external) renders the Ingress with class gce-internal, a regional static internal address (gkeIngress.staticIpName) and HTTP closed, for a regional internal Application Load Balancer that answers only inside the VPC and the networks routed to it. No ManagedCertificate or FrontendConfig is rendered (GKE Ingress supports neither internally): the certificate is gkeIngress.tlsSecretName or gkeIngress.internal.preSharedCerts (regional certificates). gkeIngress.internal.proxyOnlySubnetCidr is required and every backend NetworkPolicy (and the durable-actors policy) must admit it; the render refuses otherwise, and also refuses sslPolicy or securityPolicy in internal mode and internal.* in external mode. values-gcp-internal.yaml is the overlay; terraform/modules/gcp/internal-edge creates the proxy-only subnet, the address and the firewall rule. Default renders differ only in the helm.sh/chart labels; images unchanged. | diff --git a/self-hosted/requirements.yaml b/self-hosted/requirements.yaml index 5b696a16..115fb8e9 100644 --- a/self-hosted/requirements.yaml +++ b/self-hosted/requirements.yaml @@ -2042,9 +2042,22 @@ compatibility: images: relayauth: { repository: ghcr.io/agentworkforce/relayauth, digest: sha256:eb71daa28e30eb7305f666d0bccc5a4002e53e8a574e3c4c32163d907a1b71df } notes: superseded by 0.1.1. - - id: agent-relay-0.7.63 + - id: agent-relay-0.7.66 status: unreleased current: true + charts: { agent-relay: 0.7.66, relay-control: 0.4.22, relay-webapp: 0.2.7 } + app_versions: { agent-relay: 0.1.0, relay-control: 0.1.0, relay-webapp: 0.1.0 } + depends: + agent-relay: { relaycast: 0.3.16, relayauth: 0.1.5, sessions-pipeline: 0.8.52, relay-control: 0.4.22, relay-webapp: 0.2.7, relayflows: 0.2.5, durable-actors: 0.14.6-awf.4, relayfile: 0.2.2, telemetry-collector: 0.1.7, relay-chief: 0.1.0, slack-relay: 0.3.11, relaygate: 0.1.10 } + images: + relaycast: { repository: ghcr.io/agentworkforce/relaycast-self-host, digest: sha256:ec062ef901af15cf575b53a71d7eb09f2e6ab6e7c0d5be2da35339a275a5c6cc } + relaycast-postgres: { repository: ghcr.io/agentworkforce/relaycast-self-host, digest: sha256:ec062ef901af15cf575b53a71d7eb09f2e6ab6e7c0d5be2da35339a275a5c6cc } + sessions-pipeline-worker: { repository: ghcr.io/agentworkforce/sessions-pipeline, digest: sha256:e51749eb2b7c91b790a1e05c126cc4a7b47243ab5d8d116c62732746e223c4a5 } + relay-control: { repository: ghcr.io/agentworkforce/relay-control, digest: sha256:d1b6ca95fab240a9d41208d83de3b1f4245408b0765038762f1c419ee60f43bc } + relay-webapp: { repository: ghcr.io/agentworkforce/relay-webapp, digest: sha256:bd295ccb7d1323965dce890f19d0b7f857d706c00d7756f719c713763fdb49f3 } + notes: 'Carries 0.7.63. Opt-in EKS ALB edge: eksIngress.enabled (default false) renders the internet-facing ALB Ingress that scripts/selfhost/eks-alb-ingress.sh prints, option for option, plus eksIngress.controlPaths (default /cloud/api, /login, /relay-node). Refuses eksIngress with gkeIngress. ci/agent-relay/test_eks_alb_ingress.py proves deep equality with the script over the option matrix. Default renders differ only in chart labels; images unchanged. Not applied to any cluster.' + - id: agent-relay-0.7.63 + status: planned charts: { agent-relay: 0.7.63, relay-control: 0.4.22, relay-webapp: 0.2.7 } app_versions: { agent-relay: 0.1.0, relay-control: 0.1.0, relay-webapp: 0.1.0 } depends: