Skip to content

feat(gke-edge): internal-only mode, a regional internal load balancer (agent-relay 0.7.60) - #483

Merged
miyaontherelay merged 5 commits into
mainfrom
feat/gke-internal-ingress
Nov 19, 2026
Merged

miyaontherelay merged 5 commits into
mainfrom
feat/gke-internal-ingress

Conversation

@miyaontherelay

@miyaontherelay miyaontherelay commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #477 (base branch feat/gke-edge-armor-tls), which adds gkeIngress.tlsSecretName. Review this diff on its own; it retargets to main when #477 merges.

Why

A finding from the GCP self-host hardening review: the chart could only render an external gce Ingress with a global static address and a ManagedCertificate. A GKE install could not be kept off the internet, the way scheme: internal keeps an EKS install behind an internal ALB. This adds that mode for VPN-only installs.

What changes

Chart agent-relay 0.7.60

gkeIngress.mode: internal (default external, so default renders are unchanged apart from the helm.sh/chart labels):

External (default) Internal
Ingress class gce gce-internal (regional internal Application Load Balancer)
gkeIngress.staticIpName global external address regional internal address (kubernetes.io/ingress.regional-static-ip-name)
Certificate ManagedCertificate, or tlsSecretName tlsSecretName, or internal.preSharedCerts (regional self-managed certificates)
HTTP redirected by the FrontendConfig closed (kubernetes.io/ingress.allow-http: "false")
ManagedCertificate, FrontendConfig rendered not rendered: GKE Ingress supports neither on an internal load balancer

Hosts, routes, backends, BackendConfig timeouts and health checks are the same in both modes (a test compares the rules).

The load balancer's proxies connect to the pods from the VPC's proxy-only subnet, not from Google's front ends. So gkeIngress.internal.proxyOnlySubnetCidr is required, and every backend NetworkPolicy must admit it next to the two health-check ranges. values-gcp-internal.yaml is the overlay that sets the mode and restates the five lists; the durable-actors policy adds the range itself.

Refused combinations

Input Result
mode other than external/internal refused
internal without proxyOnlySubnetCidr, or with a non-RFC 1918 or too-small range refused
internal with no certificate, or with both tlsSecretName and preSharedCerts refused
a backend networkPolicy (relaycast, relay-control, relay-webapp, relayfile, session read API, ingest edge) without the proxy-only range, or with an except refused, naming the backend
internal with securityPolicy refused: GKE Ingress does not attach Cloud Armor to an internal load balancer
internal with sslPolicy refused: it needs a FrontendConfig, which is external-only
internal.* set while mode is external refused, so a half-applied overlay cannot pass silently

Terraform

New terraform/modules/gcp/internal-edge: the proxy-only subnet (REGIONAL_MANAGED_PROXY, or use an existing one), the regional internal address, and the firewall rule GKE does not create: from the proxy-only subnet, to instances running as the node service account, on the backend ports (3000, 7100, 8080, 8081, 8787, checked against the rendered container ports). It creates no external address and no path from outside the VPC. It does not create the certificate, because that would put the private key in state.

Docs

Chart README "Internal-only GKE edge", docs/gcp-install/3-EDGE-HARDENING.md section 4 (network, DNS, certificate, values, "Done when" from on and off the VPN), module README, registry entry opt.agent-relay.gke-internal-edge.

Decisions

  • Certificates. A Google-managed certificate is impossible here: ManagedCertificate is global-only, and GKE Ingress cannot attach Certificate Manager certificates to an internal load balancer (that needs the Gateway API, which would mean porting the whole edge). So the options are a TLS Secret, with cert-manager and DNS-01 documented as the way to get managed renewal without inbound access, or pre-shared regional certificates.
  • No TLS policy or WAF in this mode. Both are refused rather than silently ignored. The overlay clears sslPolicy/securityPolicy so that the value feat(gke-edge): TLS policy, Cloud Armor on every backend, TLS from a Secret (agent-relay 0.7.59) #477 puts in managed-values.json does not stop an internal install.
  • Both health-check ranges are still required alongside the proxy-only range.

Tests

ci/agent-relay/render.sh: annotations and absence of the external-only ones, TLS Secret and pre-shared certificate variants, no ManagedCertificate or FrontendConfig, identical rules to the external edge, every NetworkPolicy that admits the health-check range also admits the proxy-only subnet, each refusal above, the overlay clearing an earlier sslPolicy, and the values-gcp.yaml profile with the overlay. terraform test (mock provider) for the module: 4 runs.

Local: ci/agent-relay/render.sh, scripts/ci/charts.sh static agent-relay, scripts/ci/version-bump.sh, the requirements registry check, terraform fmt/validate/test, tflint, terraform-docs and checkov on the new module all pass.

Not proven

Nothing here has run on a live cluster. Unproven: that GKE syncs the Ingress as rendered (including allow-http: "false" with a SHARED_LOADBALANCER_VIP address), that the firewall rule and NetworkPolicy ranges are sufficient for traffic and health checks, the X-Forwarded-For shape of the internal load balancer (so trustedProxies is unmeasured in this mode), and long-lived WebSockets through it.

Note: render.sh has an existing yq ... | grep -q assertion ("live profile does not mount the internal token") that can fail under pipefail when grep exits early. It failed once locally on an unrelated run and passed on rerun; it is not changed here.

🤖 Generated with Claude Code


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

@miyaontherelay miyaontherelay added the merge-train:hold Do not merge without Khaliq's direct OK label Nov 18, 2026
@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 81d2683f-4fe7-446a-86ae-9f2ba464721f
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@greptile-apps

greptile-apps Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; live GKE checks are still needed before relying on it for an installation.

Summary

Adds an opt-in private GKE load balancer for VPN-only installs, with matching network setup and install guidance.

  • GKE installs can now keep the agent-relay edge inside the VPC.
  • Terraform can now prepare the network for a private GKE edge.

The numbered previous finding is fixed. No new actionable issues found. Live GKE behavior remains unproven, as documented.

Reviews (4) · Last reviewed commit: "docs(gke-edge): keep internal-mode setti..." · Reviewed by Greptile

Comment thread ci/agent-relay/render.sh Resolved
@miyaontherelay
miyaontherelay force-pushed the feat/gke-internal-ingress branch from 8f14a39 to 6700354 Compare November 18, 2026 22:53
miyaontherelay added a commit that referenced this pull request Nov 18, 2026
…s rules

Review feedback on #483:

- gkeIngress.internal.proxyOnlySubnetCidr accepted malformed text such as
  10.129.999.0/23 or 172.16.0.0.1/24, which would reach the NetworkPolicies
  and be rejected by Kubernetes at install. It must now be four octets of
  0-255 in an RFC 1918 block with a /8 to /26 prefix; refusal tests added.
- The internal/external route test compared only hosts and path counts. It
  now compares the complete spec.rules (paths, pathType, backend Services and
  ports).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@miyaontherelay
miyaontherelay changed the base branch from feat/gke-edge-armor-tls to main November 19, 2026 08:25
miyaontherelay and others added 3 commits November 19, 2026 09:25
… (agent-relay 0.7.60)

The chart could only render an external gce Ingress with a global address and
a ManagedCertificate, so a GKE install could not be kept off the internet the
way an internal-scheme ALB keeps an EKS install.

gkeIngress.mode=internal (default external):

- Ingress class gce-internal, kubernetes.io/ingress.regional-static-ip-name
  from gkeIngress.staticIpName, and allow-http "false" (HTTPS only).
- No ManagedCertificate and no FrontendConfig: GKE Ingress supports neither
  on an internal load balancer. The certificate is gkeIngress.tlsSecretName or
  gkeIngress.internal.preSharedCerts (regional self-managed certificates).
- gkeIngress.internal.proxyOnlySubnetCidr is required: the load balancer's
  proxies reach the pods from the VPC's proxy-only subnet, so every backend
  NetworkPolicy (and the durable-actors policy) must admit it.
- Refused: a missing or public proxy-only range, no certificate or both kinds,
  a backend policy without the range (named in the error), sslPolicy or
  securityPolicy in internal mode, internal.* in external mode, an unknown
  mode.

values-gcp-internal.yaml is the overlay. terraform/modules/gcp/internal-edge
creates the proxy-only subnet, the regional internal address and the firewall
rule from that subnet to the nodes' service account on the backend ports.

Docs: chart README "Internal-only GKE edge", docs/gcp-install/3-EDGE-HARDENING.md
section 4, registry entry opt.agent-relay.gke-internal-edge. Default renders
differ only in the helm.sh/chart labels.

Stacked on the edge hardening change (0.7.59), which adds tlsSecretName.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
An earlier chart ignores gkeIngress.mode and would create a public load
balancer; say so at the top of the section.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…s rules

Review feedback on #483:

- gkeIngress.internal.proxyOnlySubnetCidr accepted malformed text such as
  10.129.999.0/23 or 172.16.0.0.1/24, which would reach the NetworkPolicies
  and be rejected by Kubernetes at install. It must now be four octets of
  0-255 in an RFC 1918 block with a /8 to /26 prefix; refusal tests added.
- The internal/external route test compared only hosts and path counts. It
  now compares the complete spec.rules (paths, pathType, backend Services and
  ports).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@miyaontherelay
miyaontherelay force-pushed the feat/gke-internal-ingress branch from 7d719de to 3e70c0f Compare November 19, 2026 08:26
Comment thread docs/gcp-install/3-EDGE-HARDENING.md Resolved
miyaontherelay and others added 2 commits November 19, 2026 09:39
…pgrades carry them

Review feedback on #483: the guide added the overlay only in step 11, and the
later helm upgrade commands would drop it, switching the install back to a
public load balancer. The settings are now merged into values.yaml with yq,
and the overlay and chart README say the same for any install.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@miyaontherelay
miyaontherelay merged commit 70140b2 into main Nov 19, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-train:hold Do not merge without Khaliq's direct OK

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant