Problem
Currently the bootstrap kind cluster must remain permanently alive to host KRO/ACK controllers that manage peeks-hub and spoke clusters. This causes several issues:
- Credential expiry: The kind cluster uses EC2 instance role STS tokens (~1h) via a
aws-credentials secret. After expiry, ACK controllers lose access → PIAs stuck, EksCluster stuck IN_PROGRESS, manual task credentials:refresh required.
- Persistent EC2 dependency: The kind cluster runs inside the IDE EC2. If the IDE is restarted or SSM session expires, the kind cluster is lost and the entire management plane is gone.
- Non-idiomatic: Using a local kind cluster as the long-running control plane for production EKS clusters is fragile.
Proposed Solution: Hub Self-Adoption (CAPI Pivot Pattern)
Inspired by Cluster API pivot, the idea is to move the management of peeks-hub from the ephemeral kind cluster to the hub EKS cluster itself.
Phases
Phase 1 (today): Kind bootstrap → creates peeks-hub
kind (ephemeral) ──KRO/ACK──→ creates peeks-hub EKS
│
▼
ArgoCD/KRO/ACK (EKS Capabilities)
Phase 2 (proposed): hub:pivot → peeks-hub self-manages
peeks-hub ──KRO/ACK (in-cluster)──→ adopts its own EksCluster resource
peeks-hub ──KRO/ACK──→ manages spoke clusters
kind → destroyed (no longer needed)
Implementation Steps
- Install KRO + ACK on peeks-hub (as EKS Capabilities — already done today)
- Export EksCluster manifest from kind — serialize the current
EksCluster/peeks-hub resource
- Apply on peeks-hub context — with
adopt: true or by setting the correct owner references so ACK adopts the existing AWS resources instead of recreating them
- Delete from kind — remove the
EksCluster resource from kind (ACK will not delete the AWS resources if adopted properly)
- Pivot spoke management — move spoke
EksClusters to peeks-hub context
- Destroy kind —
kind delete cluster
Benefits
- ✅ No credential expiry (EKS PodIdentity, never temporary tokens)
- ✅ No kind cluster to maintain after bootstrap
- ✅
task destroy becomes cleaner (no kind dependency)
- ✅ Better resilience — hub EKS is the durable control plane
- ✅ Consistent with fleet model (hub manages spokes)
Challenges
- ACK adopt-or-create semantics for existing resources
- Circular dependency during bootstrap (need kind to create hub, then pivot)
- Crossplane on hub must be re-pointed to hub context for spoke management
EksCluster RGD may need adoptionPolicy support
References
Acceptance Criteria
Problem
Currently the bootstrap kind cluster must remain permanently alive to host KRO/ACK controllers that manage
peeks-huband spoke clusters. This causes several issues:aws-credentialssecret. After expiry, ACK controllers lose access → PIAs stuck, EksCluster stuck IN_PROGRESS, manualtask credentials:refreshrequired.Proposed Solution: Hub Self-Adoption (CAPI Pivot Pattern)
Inspired by Cluster API pivot, the idea is to move the management of
peeks-hubfrom the ephemeral kind cluster to the hub EKS cluster itself.Phases
Phase 1 (today): Kind bootstrap → creates peeks-hub
Phase 2 (proposed): hub:pivot → peeks-hub self-manages
Implementation Steps
EksCluster/peeks-hubresourceadopt: trueor by setting the correct owner references so ACK adopts the existing AWS resources instead of recreating themEksClusterresource from kind (ACK will not delete the AWS resources if adopted properly)EksClustersto peeks-hub contextkind delete clusterBenefits
task destroybecomes cleaner (no kind dependency)Challenges
EksClusterRGD may needadoptionPolicysupportReferences
cluster-providers/kind-kro-ack/Taskfile.yamlAcceptance Criteria
task hub:pivotmoves management from kind to peeks-hubkind delete clustersucceeds after pivot without breaking peeks-hub or spokescredentials:refreshneeded after pivottask installre-run after pivot still works (idempotent)