feat(addons): add ACK backup-controller (Preview v0.1.1) as optional addon

[allamand]

Summary

The EKS ACK capability is GA but does not currently bundle the aws-controllers-k8s/backup-controller (Preview v0.1.1, May 2026). This PR adds the backup-controller Helm chart as an optional GitOps-managed addon so users can declaratively create BackupPlan/BackupVault/BackupSelection resources from Kubernetes until AWS ships it inside the capability.

Motivation

Working on a cross-region EKS DR reference (AWS Backup + kro + ACK), I needed backup.services.k8s.aws CRDs on a PeEKS hub. Enabling the ACK capability on an EKS 1.31 cluster provisions ~180 *.services.k8s.aws CRDs, but none of the backup.services.k8s.aws group — confirmed live on eu-west-1. The only way to get them today is via the upstream Helm chart.

Changes

• gitops/addons/bootstrap/default/addons.yaml — new ack-backup entry, modeled after ack-iam and ack-eks:
  • Chart: public.ecr.aws/aws-controllers-k8s/backup-chart:0.1.1
  • Namespace: ack-system
  • Sync-wave: -1 (post-kro RGD)
  • Selector: enable_ack_backup=true
  • Same CRD ignoreDifferences as other ACK controllers
  • SA: ack-backup-controller (override via ack_backup_service_account annotation)
• platform/infra/terraform/hub-config.yaml — enable_ack_backup: false seed added to the three cluster blocks, with a comment noting the Preview status.

Out of scope (follow-up)

• IAM role / Pod Identity binding for the ack-backup-controller SA. Users must provide an ARN with AWSBackupFullAccess through the existing pod-identity annotation mechanism (same pattern as ack-iam, ack-eks).
• Flip to EKS capability once AWS includes backup-controller in the bundled ACK capability — at that point enable_ack_backup should flip back to false and the capability flag should take over.

Validation

• YAML parse OK on both files
• helm template oci://public.ecr.aws/aws-controllers-k8s/backup-chart --version 0.1.1 --namespace ack-system renders cleanly
• Deployed live on an EKS 1.31 Auto Mode hub → 3 CRDs present (backupplans, backupvaults, backupselections under backup.services.k8s.aws); controller pod starts (CrashLoops pending IRSA, as expected — out of scope here)

Related

• Upstream chart: https://github.com/aws-controllers-k8s/backup-controller (v0.1.1 PREVIEW)
• ECR Public: public.ecr.aws/aws-controllers-k8s/backup-chart:0.1.1

[allamand]

Update: Pod Identity wiring via kro + ACK

Second commit (795c45c1) completes the PR by wiring the backup-controller's Pod Identity via the existing kro + ACK path (not Crossplane).

Added

• gitops/addons/charts/application-sets/templates/_pod_identity_kro.tpl — new template, mirrors _pod_identity.tpl but points at gitops/addons/charts/kro/instances/pod-identity (the existing kro-pi-instance chart backed by the podidentity.kro.run RGD).
• Second guard in application-set.yaml triggered by enableACKKRO: true. Mutex with enableACK: setting both fails the helm template with an explicit error.
• podIdentity block on ack-backup: namespace, service account, and least-privilege policyDocument.

IAM policy scope

• backup:*, backup-storage:*
• iam:PassRole scoped to arn:aws:iam::*:role/* (decoupled from backup-role naming)
• KMS: Decrypt, Encrypt, GenerateDataKey*, DescribeKey, CreateGrant — required for cross-account encrypted snapshot copies.

Validation

• helm template renders 3 Argo sources for ack-backup (values / pod-identity / chart) ✅
• Mutex triggers fail as expected ✅
• No change to existing addons using enableACK (Crossplane path untouched) ✅

Resolves the IRSA CrashLoopBackOff observed after deploying ack-backup-controller Helm chart standalone on an EKS Auto Mode cluster.

[allamand]

Commit 3 3d93e6bc — Terraform: extend ACK capability role for kro+ACK Pod Identity path

Root cause discovered during end-to-end validation on hub eu-west-1: the upstream aws_iam_role.eks_capability_ack role (in platform/infra/terraform/cluster/main.tf) was missing the IAM permissions needed by the kro+ACK pod-identity pattern introduced in commit 2. Symptom: kro PodIdentity CR stuck — ACK iam.Policy / iam.Role / PodIdentityAssociation reconciliation failed with AccessDenied.

Gaps (confirmed via runtime testing):

• iam:CreatePolicy / GetPolicyVersion / CreatePolicyVersion / ... — the kro RGD creates a managed iam.services.k8s.aws/Policy (not inline on the role), so ManageIRSARoles perms were insufficient
• iam:PassRole — needed so eks.services.k8s.aws/PodIdentityAssociation can pass the workload role to the EKS Pod Identity service
• eks:*PodIdentityAssociation* + eks:TagResource/UntagResource/ListTagsForResource — the PodIdentityAssociation CRUD itself
• ManageIRSARoles filter environment != "control-plane" — excluded the hub, blocking kro+ACK-wired addons on the capability cluster

Fix (3 policies on aws_iam_role.eks_capability_ack):

1. ManageIRSARoles — removed control-plane filter + clarifying comment (IAM actions are trust-policy-agnostic, same perms work for IRSA and Pod Identity)
2. ManageWorkloadPolicies (new) — iam:CreatePolicy/DeletePolicy/GetPolicy/GetPolicyVersion/ListPolicyVersions/CreatePolicyVersion/DeletePolicyVersion/TagPolicy/UntagPolicy scoped to policy/${cluster.name}-*
3. ManagePodIdentityAssociations (new) — iam:PassRole (condition iam:PassedToService = pods.eks.amazonaws.com, scoped to role/${cluster.name}-*) + eks:*PodIdentityAssociation* scoped to cluster/${cluster.name} + podidentityassociation/${cluster.name}/*

Validation end-to-end on hub eu-west-1 (account 586794472760), workarounds removed:

• kro PodIdentity CR kro-pi-instance-ack-backup-controller → STATE=ACTIVE READY=True
• ACK Policy hub-backup-chart-pod-policy → SYNCED=True (ARN resolved)
• ACK Role hub-backup-chart-role → SYNCED=True (trust pods.eks.amazonaws.com)
• ACK PodIdentityAssociation → SYNCED=True, aws eks list-pod-identity-associations returns the association
• ack-backup-controller-backup-chart pod → Running 1/1 (was CrashLoopBackOff)
• BackupVault CR → AWS Backup vault created, deleted cleanly

terraform plan diff: 3 to add, 0 to change, 0 to destroy (one per cluster key — spoke clusters already had ManageIRSARoles, this just adds the 2 new Pod-Identity policies; hub gets all 3).