Skip to content

fix(edge-security): Cloud Armor rejects capture groups; rule counts, quotas, 10 subexpressions - #487

Merged
kjgbot merged 11 commits into
mainfrom
fix/edge-armor-quota
Nov 19, 2026
Merged

kjgbot merged 11 commits into
mainfrom
fix/edge-armor-quota

Conversation

@miyaontherelay

@miyaontherelay miyaontherelay commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Why

Follow-up to #477 (merged), from the live GCP proof that layered-infra-ops is running. The first item is a blocker in the module on main.

  1. Cloud Armor rejects capture groups (blocker). Every client_ip_rules expression used request.path.matches('^(a|b)') and a host pattern with groups. The API refuses them: Invalid regular expression ... Capture Groups are not allowed. So any policy with client_ip_rules fails at apply on main today. The default sign-in rate-limit regex had the same form. The mock-provider tests could not catch this because nothing sends an expression to the API.
  2. A vendor list exceeds Google's default quotas. A new project allows 20 rules with an expression (SECURITY_POLICY_CEVAL_RULES) and 100 rules in all (SECURITY_POLICY_RULES). A host-scoped rule with about 290 vendor ranges planned about 105 and 109, and nothing said so before the apply.
  3. The per-list limit was too tight: 294 ranges, with the vendor list at 292.
  4. A missing API: the Terraform root needs cloudresourcemanager.googleapis.com for data.google_project; without it plan fails with a 403.
  5. Empty-project path: gcloud container clusters create fails in an organization that forbids the default Compute Engine service account for nodes.

What changes

  • Non-capturing groups everywhere. Generated path and host patterns use (?:...), the form the API accepted in the proof. The default rate_limits regex is fixed, and a capture group in a user's path_regex is refused at plan with the reason. A test asserts that no generated expression contains a capture group.
  • max_subexpressions, default 10. Measured: the API accepts 10 subexpressions in one expression and refuses 11 ("Expression count of 11 exceeded maximum of 10"). A rule now carries 9 client ranges, or 8 with hosts, where it carried 4 or 3. The 290-range list needs about 45 advanced rules, not about 105.
  • rule_counts output (advanced, total) on the module and in terraform/examples/gcp output.edge, so terraform plan shows what the policy needs against the quotas.
  • Docs: "Quotas: check before you apply" in the module README and the same warning in docs/gcp-install/3-EDGE-HARDENING.md (see the per-policy limit below).
  • Priority bands of 1000 (client rules 10000+, rate limits 20000+, allowlist 30000+): a list can hold 2,500 ranges and 999 rules. Relative order is unchanged.
  • Guide: cloudresourcemanager.googleapis.com in step 2's gcloud services enable; a node service account for organizations that forbid the default one.

Effect on existing policies

Expressions and priorities change, so a policy applied from #477 has its client, rate-limit and allowlist rules replaced on the next apply. No such policy can exist with client_ip_rules (it could not be created), and the only known policy is the proof stack's.

Tests

terraform test (mock provider), 34 runs in the module and 12 in the example root: the exact host and path patterns the API accepted; no capture group in any generated expression; a capture group in path_regex refused, (?:...) and escaped parentheses accepted; rule_counts equals the number of rule resources; 8 ranges per host-scoped rule at 10 subexpressions; the Cloudflare default needs 5 advanced rules and 8 in all. Local: terraform fmt/validate/test, tflint, terraform-docs and checkov pass. No chart change, so no version bump.

The per-policy limit (added after the first live apply)

After the project quota was raised to 100, the full policy still failed: Quota 'SECURITY_POLICY_ADVANCED_RULES_PER_SECURITY_POLICY' exceeded. Limit: 20. That quota is fixed; a request to raise it is refused. One policy holds at most 20 rules with an expression. So:

  • A precondition (max_advanced_rules, default 20) now refuses a plan that needs more, before anything is created, with the count and the options.
  • With the secret header, two WAF rules and sign-in and staff rules, about 13 rules remain for a vendor list: roughly 100 ranges. A 293-range list cannot fit in this policy at any setting. The docs say to merge ranges where the wider block is acceptable, or enforce that vendor's list at the fronting proxy. Address groups (Cloud Armor Enterprise) are not supported by this module.
  • The docs also quote the project-quota error, say that it is granted automatically in seconds when requested (measured on an existing project), and warn that a failed apply leaves a partial policy that must not be attached.

A caller with the secret could skip the proxy (added after the policy was attached live)

With the policy attached, the proof found that a request admitted by a client rule never reaches the allowlist of connecting addresses, so the shared-secret header was the only guard. A caller holding the secret, connecting straight to the load balancer, was admitted and chose its own client address through CF-Connecting-IP: as a staff address, and as a vendor on a vendor path.

  • New rule at priority 150, ahead of the WAF and of every rule that admits: deny a connection that comes from none of connecting_asns (Cloudflare: 13335) or connecting_address_groups (an address group of the proxy's ranges, Cloud Armor Enterprise). The proxy's 15 ranges do not fit one negated expression.
  • client_ip_rules are refused at plan without one of them, unless secret_header_only = true accepts the header as the sole guard.
  • Path prefixes now end at a path segment (/mcp no longer matches /mcpfoo).
  • Guide: the backend-services list showed the policy about 4 minutes before every backend enforced it, so the check now sends a request per host. trustedProxies: 3 behind Cloudflare is now measured.
  • terraform/examples/gcp: labels on the private Cloud SQL DNS zone.

Proven live vs not

Proven on the proof stack with the policy attached (at commit b39e76a): the non-capturing patterns, 10 subexpressions, a clean apply at exactly 20 advanced rules, both quota errors, the secret header rule, vendor path scoping (admitted on the listed paths, denied on 12 others), the sign-in limit (60 then 429), throttles counting per CF-Connecting-IP, WAF preview logging, hosted MCP end to end.

Proven on a second live run (at commit 60afa9a, connecting_asns = [13335], Standard tier): rule 150 denied 31 of 31 direct requests that carried the secret, from three vantage points, whatever CF-Connecting-IP claimed; 0 of 668 requests through Cloudflare were decided by it (all arrived from AS13335; two Cloudflare locations sampled, so not every location is proven). The path-segment boundary: /mcp, /mcp/x admitted; /mcpfoo, /mcp.json, /MCP and encoded-slash tricks denied. A plan after the apply shows no changes.

Not proven: the address-group form of the guard (needs Cloud Armor Enterprise). Not run, because the proof's Cloudflare token cannot edit rules or certificates: the Cloudflare transform rule that adds the secret header (the client supplied it in the tests), Full (strict) with an Origin CA certificate, and a connection held open past 100 seconds.

Known limits, documented: a new rule takes up to about 4 minutes to be enforced on every backend; the connecting check costs one of the 20 rules; another customer of the same proxy who holds the secret still arrives from the same network.

🤖 Generated with Claude Code

…as, lift the 294-range limit

Found by the live GCP proof at plan time, before any apply: a client rule with
a vendor's published ranges (about 290) plans about 105 rules with an
expression and 109 rules in all, against Google's defaults of 20
(SECURITY_POLICY_CEVAL_RULES) and 100 (SECURITY_POLICY_RULES) per project. The
apply would fail part-way with a quota error, and nothing said so beforehand.

- New output rule_counts (advanced, total), also in examples/gcp output.edge,
  so terraform plan shows what the policy needs.
- Module README "Quotas: check before you apply" and the GKE edge guide:
  request the two quotas first, or enforce a long list at the fronting proxy.
- max_subexpressions (default 5, unchanged behaviour): how many ranges fit in
  one rule. Google's quota page lists 10, which would need about a third as
  many rules; not proven, so opt-in.
- A rule list was capped at 294 ranges by the priority spacing, and the vendor
  list was at 292. Rule priorities now use bands of 1000 (client rules 10000+,
  rate limits 20000+, allowlist 30000+): up to 999 rules and 2500 ranges per
  list. Relative order is unchanged. An existing policy's rules are replaced
  on the next apply.
- The guide's API list gains cloudresourcemanager.googleapis.com, which the
  Terraform root needs for data.google_project (plan otherwise fails 403).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@miyaontherelay miyaontherelay added the merge-train:hold Do not merge without Khaliq's direct OK label Nov 19, 2026
@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2f69d19f-8663-4354-84dd-248775b998e2
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@greptile-apps

greptile-apps Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[High impact] This PR appears safe to merge.

Summary

This PR fixes Cloud Armor expressions, packs address lists into fewer rules, and adds checks for rule limits and direct connections that skip the proxy.

  • Cloud Armor rules use API-safe patterns and show their quota cost.
  • Client rules can check that connections come from the fronting proxy.
  • The GCP setup guide covers a required API and a custom node account.
  • The private Cloud SQL DNS zone receives the example's labels.

The earlier findings are addressed. No new actionable issues were found.

Reviews (10) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." · Reviewed by Greptile

…apture groups); 10 subexpressions

Measured against the Cloud Armor API by the live GCP proof:

- BLOCKER in the merged module: every client_ip_rules expression used
  request.path.matches('^(a|b)') and a host pattern with groups. Cloud Armor
  refuses them: "Invalid regular expression ... Capture Groups are not
  allowed", so each client allow and deny rule would fail at apply. All
  generated patterns now use (?:...), the form the API accepted. The default
  sign-in rate-limit regex had the same problem and is fixed, and a capture
  group in rate_limits[*].path_regex is refused at plan. A test asserts no
  generated expression contains one.
- The API accepts 10 subexpressions per expression and refuses 11.
  max_subexpressions now defaults to 10: nine client ranges per rule, eight
  with hosts. A 290-range vendor list needs about 45 advanced rules instead
  of about 105; still above the default quota of 20, so the quota request
  stays a documented prerequisite.
- Guide, empty-project path: what to do when the organization forbids the
  default Compute Engine service account for nodes.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@miyaontherelay miyaontherelay changed the title fix(edge-security): report Cloud Armor rule counts, document the quotas, lift the 294-range limit fix(edge-security): Cloud Armor rejects capture groups; rule counts, quotas, 10 subexpressions Nov 19, 2026
Review feedback on #487: the list fixtures had 24 ranges and would still
pass with the old 294-range cap or 100-wide priority spacing. Adds a
host-scoped list of 300 ranges (100 admitting rules) followed by another
client rule, asserting unique priorities, every allow before its deny, the
deny before the next rule, and the reported rule count.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
miyaontherelay and others added 3 commits November 19, 2026 12:00
Review feedback on #487: the check refused any remaining opening
parenthesis, so valid patterns such as (?i)^/login and ^/file[(]old[)]$ failed
plan. It now drops escaped characters and character classes first and refuses
only a group that captures: "(" not followed by "?", or a named group. Tests
cover flags, bracketed and escaped parentheses, a named group and a capture
group after a class.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…after a failed apply

Measured by the live proof at b39e76a: plan gives no quota warning; apply
fails with Quota 'SECURITY_POLICY_CEVAL_RULES' exceeded (limit 20) and leaves
an arbitrary subset of rules, including a sign-in deny without its allow.
The README and guide now quote the error and say to attach the policy only
after an apply that exits cleanly.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… rules per policy

Measured by the live proof: after the project quota was raised to 100, the
full policy still failed, 0 of 25 rules created, with
Quota 'SECURITY_POLICY_ADVANCED_RULES_PER_SECURITY_POLICY' exceeded. Limit: 20.
That quota is fixed (a request to raise it is refused), so one policy holds at
most 20 rules with an expression whatever the project allows.

- A precondition (max_advanced_rules, default 20) refuses such a plan before
  anything is created, with the count and what to do.
- README and guide: the two limits, which one can be raised (the project one,
  granted automatically in seconds when measured), and that about 100 vendor
  ranges is what fits beside the secret header, WAF, sign-in and staff rules.
  A longer list has to be merged into wider ranges or enforced at the proxy.
- Tests: 293 host-scoped ranges are refused at plan; the 96-range shape the
  proof applied cleanly needs exactly 20.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ient rules; path prefixes end at a segment

Findings from the live proof with the policy attached (S3-d and smaller ones):

- SECURITY: a request admitted by a client rule never reaches the allowlist of
  connecting addresses, so the shared-secret header was the only guard. A
  caller holding the secret, connecting straight to the load balancer, was
  admitted and chose its own client address through CF-Connecting-IP (as a
  staff address, and as a vendor on a vendor path). New rule at priority 150
  denies a connection that comes from none of connecting_asns (Cloudflare:
  13335) or connecting_address_groups (an address group of the proxy's ranges,
  Cloud Armor Enterprise); the proxy's 15 ranges do not fit one negated
  expression. client_ip_rules are now refused without one of them, unless
  secret_header_only = true accepts the header as the sole guard.
- A path prefix had no segment boundary (/mcp also matched /mcpfoo). Patterns
  now end in (?:/|$); "/" stays every path.
- Guide: listing the policy on all backend services passed about 4 minutes
  before every backend enforced it; the check now sends a request per host.
  trustedProxies 3 behind Cloudflare is measured, not expected.
- examples/gcp: labels on the private Cloud SQL DNS zone.

Proven live before this commit: the capture-group fix, 10 subexpressions,
throttle keyed per CF-Connecting-IP, the sign-in limit, WAF preview logging,
vendor path scoping. NOT yet proven: origin.asn in a deny rule and the
address-group form of the guard.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Comment thread docs/gcp-install/3-EDGE-HARDENING.md Resolved
miyaontherelay and others added 2 commits November 19, 2026 13:33
…two README notes

Review feedback on #487: the per-host loop only requested / on the history
host, which reaches the deny service, so the history reader and upload
backends were never probed. The check now sends one request to each of the six
backend services. README: a trailing slash on a path prefix is ignored, the
connecting check counts as a rule, and rule_counts shows the plan, not what
exists after a failed apply.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… lag and the remaining gap

Measured on the proof stack at 60afa9a with connecting_asns = [13335]: 31 of
31 direct requests carrying the secret were denied by rule 150, and none of
668 requests through Cloudflare were. The guide now says to poll a direct
request with the header until every backend refuses it (a new rule took
almost 4 minutes), and names the gap that remains: another customer of the
same proxy who holds the secret.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Comment thread docs/gcp-install/3-EDGE-HARDENING.md Resolved
miyaontherelay and others added 2 commits November 19, 2026 14:12
…d client address

Review feedback on #487: with the secret header alone, a test machine outside
the client ranges gets 403 from the client rules themselves, so the probe
passed before the connecting check was enforced. It now also sends
CF-Connecting-IP with an address the staff rule admits, which is the bypass
the check closes.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@kjgbot kjgbot removed the merge-train:hold Do not merge without Khaliq's direct OK label Nov 19, 2026
@kjgbot
kjgbot merged commit 311f5de into main Nov 19, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants