Skip to content

feat(gke-edge): TLS policy, Cloud Armor on every backend, TLS from a Secret (agent-relay 0.7.59) - #477

Merged
miyaontherelay merged 12 commits into
mainfrom
feat/gke-edge-armor-tls
Nov 19, 2026
Merged

miyaontherelay merged 12 commits into
mainfrom
feat/gke-edge-armor-tls

Conversation

@miyaontherelay

@miyaontherelay miyaontherelay commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Why

Two findings from the GCP self-host hardening review:

  1. TLS policy. The GKE Ingress FrontendConfig only set redirectToHttps, so Google's default SSL policy applied and the load balancer accepted TLS 1.0 and 1.1. The EKS path pins ELBSecurityPolicy-TLS13-1-2-2021-06.
  2. No edge WAF. No backend had a Cloud Armor policy.

It also adds what an install behind a fronting proxy (Cloudflare) needs: a certificate from a Secret, and a policy that only admits the proxy.

What changes

Chart agent-relay 0.7.59

Three new gkeIngress values, all empty by default. Default renders are unchanged apart from the helm.sh/chart labels.

Value Effect
gkeIngress.sslPolicy Compute SSL policy name → FrontendConfig.spec.sslPolicy
gkeIngress.securityPolicy Cloud Armor policy name → spec.securityPolicy.name on the BackendConfig of every backend behind the Ingress
gkeIngress.tlsSecretName kubernetes.io/tls Secret in spec.tls instead of the ManagedCertificate (not rendered)

"Every backend" means: api, cast, history read, history ingest, actors edge and relaygate edge (their existing BackendConfigs keep timeouts and health checks), plus new policy-only BackendConfigs <name>-dashboard, <name>-edge-deny and <name>-files. The dashboard and Relayfile Services belong to child charts, so the render refuses until their cloud.google.com/backend-config annotation names the derived BackendConfig, and prints the value to set. No child chart changes.

Terraform

  • New terraform/modules/gcp/edge-security:
    • google_compute_ssl_policy (MODERN or RESTRICTED, TLS 1.2 minimum; COMPATIBLE/TLS 1.0 are refused by validation).
    • Optional Cloud Armor policy (security_policy.create): OWASP preconfigured rules at sensitivity 1, a per-client rate limit on sign-in paths, a source-IP allowlist with default deny (also usable alone, as a pure allowlist), Cloudflare origin lockdown (published ranges only), a shared-secret origin header rule, and path rules keyed on the proxy's client-IP header (origin.user_ip). Labels on the policy.
  • terraform/examples/gcp calls it: the SSL policy is created by default and exported as gkeIngress.sslPolicy in chart_values; Cloud Armor is opt-in (edge_security_policy).

Docs

docs/gcp-install/3-EDGE-HARDENING.md (steps and "Done when" checks for TLS policy, Cloud Armor, Cloudflare in front, including trustedProxies: 2 and which hostnames a Cloudflare Universal certificate covers), chart README "GKE edge hardening", module README, GKE skill inputs, registry entry opt.agent-relay.gke-edge-hardening.

Defaults chosen, and why

  • gkeIngress.sslPolicy: empty in the chart, on by default through Terraform. The chart cannot create a Compute resource, and a name that does not exist in the project stops the Ingress from syncing, so a chart default would break every install that upgrades. Instead terraform/examples/gcp creates the policy and puts its name in managed-values.json, which the guide already passes to helm upgrade. New guided installs get TLS 1.2+ with no extra step; an existing install gets it the next time it re-applies step 4 and upgrades (plan shows one resource to add; the documented plan count goes from 29 to 30). Opt out with edge_ssl_policy = { create = false }. Bring-your-own installs create it with one gcloud command.
  • Cloud Armor: off. As asked. It needs the customer to review the rules against their traffic first.
  • OWASP rules: preview by default. Transcripts, messages and flows carry code, SQL and shell text, which is what those signatures match. Preview logs without blocking; the guide gives the log query and the switch to enforce.
  • Sensitivity 1, and methodenforcement plus the language-specific rule sets are left out (the API uses PUT/PATCH/DELETE).

Design notes for review

  • Cloud Armor expressions take five subexpressions, and a throttle rule admits conforming requests. So when an allowlist is set, each throttle rule also requires an allowlisted connecting address, in groups of four ranges. Deny rules always precede rules that admit.
  • Cloudflare's ranges are shared by all Cloudflare customers and cannot be conjoined with a second address list in one expression, so client_ip_rules (which trust CF-Connecting-IP) require origin_secret_header. That value is in Terraform state and readable in the policy; the docs say so.
  • 0.7.58 is taken by feat(relay-control): deployment default sharing mode and Control 0.5.35 (relay-control 0.4.23, agent-relay 0.7.68) #474; this PR uses 0.7.59 and will be rebased on whichever lands first.

Tests

  • ci/agent-relay/render.sh: defaults render none of the three; with them set, the FrontendConfig carries sslPolicy and keeps the redirect, every BackendConfig carries the policy, every Service the Ingress routes to is bound to a BackendConfig that has it, timeouts and health checks survive; actors/files overlay; tlsSecretName drops the ManagedCertificate and annotation; each refusal.
  • terraform test (mock provider): 10 runs in the module, 3 new runs in examples/gcp.
  • Local: scripts/ci/charts.sh static agent-relay, ci/agent-relay/render.sh, scripts/ci/version-bump.sh, node ci/self-hosted-requirements/registry.mjs check, terraform fmt/validate/test, tflint, terraform-docs, checkov on the changed module and example: all pass. Local Helm is 4.1.3, so the subdomain golden hash check was skipped locally and runs in CI.

Not proven

Nothing here has run against a live GCP project: no terraform apply, no real load balancer. Unproven until a live install: that GKE attaches the policies as rendered, the Cloud Armor rule expressions and quotas (seven advanced rules with Cloudflare on), trustedProxies: 2 behind Cloudflare, and long-lived WebSocket/SSE connections through Cloudflare.

🤖 Generated with Claude Code


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

…Secret (agent-relay 0.7.59)

The GKE Ingress's FrontendConfig only set the HTTPS redirect, so Google's
default SSL policy applied (TLS 1.0 and later), and no backend had a Cloud
Armor policy.

Chart (all three values empty by default; default renders unchanged apart
from the helm.sh/chart labels):

- gkeIngress.sslPolicy: Compute SSL policy name, rendered as
  FrontendConfig spec.sslPolicy.
- gkeIngress.securityPolicy: Cloud Armor policy name, set on the
  BackendConfig of every backend behind the Ingress (api, cast, both history
  edges, actors edge, relaygate edge) plus new policy-only BackendConfigs for
  the dashboard, the edge-deny Service and Relayfile. The dashboard and
  Relayfile Services are child-chart objects, so the render refuses until
  their backend-config annotation names the derived BackendConfig.
- gkeIngress.tlsSecretName: a kubernetes.io/tls Secret instead of the
  ManagedCertificate, for an edge behind a proxy or CDN.

Terraform:

- modules/gcp/edge-security: google_compute_ssl_policy (MODERN or
  RESTRICTED, TLS 1.2 minimum) and an optional Cloud Armor policy: OWASP
  preconfigured rules (preview by default), per-client sign-in rate limit,
  source-IP allowlist with default deny, Cloudflare origin lockdown, a
  shared-secret origin header rule and client-address path rules keyed on
  the proxy's client-IP header. Labels on the policy.
- examples/gcp creates the SSL policy by default and exports it as
  gkeIngress.sslPolicy in chart_values; Cloud Armor is opt-in.

Docs: docs/gcp-install/3-EDGE-HARDENING.md, chart README, module README,
registry entry opt.agent-relay.gke-edge-hardening.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@miyaontherelay miyaontherelay added the merge-train:hold Do not merge without Khaliq's direct OK label Nov 18, 2026
@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 80c56225-02c5-4e1c-8c30-2e3e08585527
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kjgbot

kjgbot commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

Reviewed against the f6b9x4unfg GKE POC values (13 -f files, chart currently packaged from main 42e534c). Render only; nothing applied.

Render

  • With the edge settings off, this branch renders the same as the live chart, apart from the search-indexer bootstrap Job hash and main drift. That makes it a safe no-op upgrade.
  • With sslPolicy, securityPolicy, tlsSecretName and the relay-webapp backend-config annotation set, the rendered result is as intended:
    • ManagedCertificate and its Ingress annotation are dropped, and spec.tls points at the Secret;
    • securityPolicy lands on every BackendConfig: api, cast, gate, actors, history read and ingest, plus the new dashboard and edge-deny configs;
    • sslPolicy is set in the FrontendConfig.
  • The render fails with the exact value to set when the relay-webapp annotation is missing. That's good.

1. trustedProxies: 2 behind Cloudflare is likely one short (please verify on a live LB before documenting)

  • clientIp() in relay-control takes the Nth entry from the right of X-Forwarded-For.
  • The Google external Application LB appends two entries, <address it saw>, <LB IP>, not one.
    • Behind Cloudflare the header reaches the pod as client, cloudflare-edge, lb-ip. N=2 returns the Cloudflare edge, so every user shares a few limiter keys and the password lockout. N=3 returns the client.
    • Without Cloudflare the same reasoning gives 2, not 1. Several installs (this POC included) run trustedProxies: 1, which would key on the LB IP. If that holds, it's a separate existing bug worth its own issue.
  • Suggest: log x-forwarded-for once on a test request through each path and document the measured number.

2. Rotating the origin secret header causes an outage

  • The rule accepts exactly one value. Changing it in Terraform and in the Cloudflare Transform Rule can't happen atomically, so requests 403 in between.
  • Consider accepting a list (values = [current, next], joined with ||), so rotation is: add next, switch Cloudflare, drop old.

3. Docs: client_ip_rules vs Cloudflare WAF

  • The module's CF-Connecting-IP client rules are correct, because the precondition requires the secret header.
  • For the Cloudflare case I'd still point readers at Cloudflare WAF custom rules for per-client and per-path allowlists. Cloudflare sees the true client and keeps the rule count off Cloud Armor's expression limits.
  • Keep the Cloud Armor rules for installs fronted by something other than Cloudflare.

4. Small

  • ssl_policy has no labels in the API (noted). The forwarding rules and backends that GKE Ingress creates also take none. Worth a line in 3-EDGE-HARDENING.md for customers who require billing labels on everything: GKE cost allocation covers in-cluster spend.
  • The Cloudflare CIDR defaults are pinned in the variable. Add a dated comment and a note to re-check https://www.cloudflare.com/ips/ at each apply.

No blockers from the POC side apart from item 1, which should be measured before the docs state a number.

@greptile-apps

greptile-apps Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The documentation fix appears safe to merge; no blocking issues remain.

Summary

The latest change fixes the TLS policy removal steps by detaching the policy before Terraform deletes it.

  • The GKE HTTPS listener can use a TLS 1.2-or-higher policy.
  • Cloud Armor can filter requests across every GKE Ingress backend.
  • The GKE Ingress can serve a TLS Secret instead of a ManagedCertificate.

No new issues found. Earlier fixes remain in place. miyaontherelay accepted that approved client rules skip later source-address checks because the required secret header proves the request came through the proxy; each rule must state its own rate limit or explicitly allow unlimited requests.

Reviews (10) · Last reviewed commit: "docs(gke-edge): detach the SSL policy th..." · Reviewed by Greptile

…nce, Cloudflare WAF pointer

Review feedback on #477:

- origin_secret_header takes values (1-3 accepted values) instead of one
  value, so rotation is add-next, switch the proxy, drop-old with no 403
  window.
- The guide no longer states trustedProxies: 2 behind Cloudflare. The Google
  load balancer appends two X-Forwarded-For entries, so the expected value is
  3; the guide says to raise the measured value by one and how to measure it
  from the audit log.
- Per-client path rules behind Cloudflare: point at Cloudflare WAF custom
  rules; client_ip_rules stay for other fronting proxies.
- Note which resources carry labels, and to compare Cloudflare's published
  ranges before each apply.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@miyaontherelay

Copy link
Copy Markdown
Contributor Author

Thanks for the review against the POC values. Addressed in 34090c4:

  1. trustedProxies: agreed, and I had no measurement. The guide no longer states 2. It now says the right value is one more than the value that is correct without Cloudflare, explains that the Google load balancer appends two X-Forwarded-For entries (so 3 is expected), and gives a way to measure it from the audit log. The trustedProxies: 1 already in docs/gcp-install/values.yaml is outside this PR; I'll raise it with the infra lead as a separate issue to be measured on a live load balancer.
  2. Secret rotation: origin_secret_header.values now takes 1-3 accepted values. Rotation is add the next value, switch the Cloudflare rule, remove the old one. Tests cover two values and refuse four.
  3. Cloudflare WAF: the guide and module README now point Cloudflare users at Cloudflare WAF custom rules for per-client and per-path rules; client_ip_rules remain for other fronting proxies.
  4. Small: added the labels note (SSL policy and GKE-created forwarding rules/backends have none; GKE cost allocation) and compare with cloudflare.com/ips before each apply. The dated comment was already on the variable.

…ured trustedProxies

- client_ip_rules take optional hosts and are documented as the way to say
  which clients may reach which paths behind a proxy: rules apply in list
  order, and a closing ["/"] rule shuts every other path. Test covers vendor
  ranges on the history upload paths plus a closed remainder. Cloudflare WAF is
  the alternative when a vendor's list exceeds Cloud Armor's advanced-rule
  quota.
- trustedProxies: measured on a live GKE install, the Google load balancer
  appends two X-Forwarded-For entries, so the value is 2 without a fronting
  proxy and 3 behind Cloudflare (expected, not yet measured). The guide shows
  how to read the header from the relay-webapp access log.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…l-safe validation

Review feedback on #477:

- A client_ip_rules entry admitted its clients with a terminal allow at
  priority 2000+, so those requests skipped the rate limits (and the
  connecting-address allowlist). A rule whose ranges fit one expression (4,
  or 3 with hosts) is now a single deny rule ("these paths and not one of
  these clients"): it admits nobody, and its clients continue to the rate
  limits and allowlist. A longer list cannot be negated across rules, so it
  still admits; it can carry its own per-client rate_limit (throttle keyed on
  the client address), and the module refuses a long rule with neither a
  rate_limit nor unthrottled = true while rate_limits is set.
- origin_secret_header validation used "== null ||" around attribute reads,
  which fails on null before Terraform 1.9 (both sides are evaluated). It is
  now a conditional.
- Guide: raise trustedProxies for Cloudflare only after direct requests to
  the load balancer are refused; otherwise a direct caller can forge its
  address.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
miyaontherelay and others added 2 commits November 18, 2026 23:42
…s own limit

Second review round on #477. The deny-only form for short lists let an
approved client fall through to later client rules (a vendor admitted by its
own rule was then denied by a closing "/" rule), and it ignored a rule's
rate_limit.

client_ip_rules now have one form and one meaning: the first rule whose paths
(and hosts) match decides the request. Its clients are admitted, by a throttle
keyed on the client address when the rule has a rate_limit, and everyone else
on those paths is denied. Because an admitted request meets no later rule:

- every rule must set rate_limit or unthrottled = true;
- a rule for every path ("/" without hosts) is refused while rate_limits is
  non-empty, since those limits could never apply; the sign-in limit is then
  written as a narrower rule ahead of it.

Tests cover a short vendor list ahead of a staff-only "/" rule, a one-range
rule with a rate_limit, and both refusals.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… in vendor rules

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ay which release they need

The GKE guide installs agent-relay-2026.11.16 (chart 0.7.54, Terraform from
that tag), which has neither the gkeIngress edge keys nor the edge_* Terraform
inputs, and its chart would ignore the keys without an error.

- 0-START-HERE.md: step 4 keeps the pinned release's resource list and plan
  count (29); step 14 points at the hardening guide instead of claiming the
  TLS policy exists.
- 3-EDGE-HARDENING.md and the values template: these controls need chart
  0.7.59 or later and the Terraform from the same release, with a command to
  check.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@miyaontherelay

Copy link
Copy Markdown
Contributor Author

Self-review finding, fixed in c0c043f: the GKE guide installs the pinned release agent-relay-2026.11.16 (chart 0.7.54, Terraform from that tag), which has none of this PR's chart keys or Terraform inputs, and its chart would ignore the keys without an error. I had changed step 4's plan count to 30 and said the TLS policy was already in place after step 4; both were wrong for the pinned release. Step 4 is back to the pinned release's text, and 3-EDGE-HARDENING.md and the values template now say these controls need chart 0.7.59 or later with the Terraform from the same release. So "on by default through Terraform" in the description holds from the first release that carries chart 0.7.59, not for the release the guide pins today.

miyaontherelay and others added 2 commits November 19, 2026 08:54
…e expression

Review feedback on #477: path_regex went into the quoted Cloud Armor
expression as written, so a regex escape such as \. was consumed by the
expression's string reader and the rule failed to apply. Backslashes are now
doubled, as they already are for client rule paths and hosts. Test added.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Comment thread docs/gcp-install/3-EDGE-HARDENING.md Resolved
Review feedback on #477: GKE leaves a backend's security policy in place when
spec.securityPolicy is absent and detaches only on an empty name, so clearing
the chart value did not detach it and Terraform could not delete the policy.
The removal steps now patch every BackendConfig to an empty name, wait, then
clear the values and destroy.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Comment thread docs/gcp-install/3-EDGE-HARDENING.md Resolved
…aform removes it

Review feedback on #477: managed-values.json keeps the old sslPolicy until an
apply, and the apply cannot delete a policy still in use. The steps now clear
it with an override in values.yaml, wait, then apply and regenerate.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-train:hold Do not merge without Khaliq's direct OK

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants