Skip to content

docs: explain chart release automation #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
xnoto merged 1 commit into from
Aug 28, 2026
Merged

docs: explain chart release automation #15

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 10 additions & 5 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,16 +36,21 @@ sensitive plan output.

## Chart conventions

- Use [Adding a chart](docs/adding-a-chart.md) for the authoring and release
procedure. Use [GitOps update automation](docs/gitops-update-automation.md)
for the optional post-publish pull request contract.
- Keep each chart self-contained below its chart directory.
- Do not place plaintext credentials, kubeconfigs, tokens, or decrypted SOPS
values in charts or values files.
- Cluster-owned secrets, namespaces, storage, and shared networking remain in
`kustomize-cluster` unless a migration explicitly moves their ownership.
- Update the chart README and CI coverage when adding a chart.
- Add `<chart>/README.md` for chart-specific ownership, values, prerequisites,
and usage; update it when that contract changes. CI discovers direct-child
charts automatically.
- `opencode-server/files/opencode.json` and `opencode-server/files/agents/*.md`
are rendered into the chart ConfigMap. Bump `opencode-server/Chart.yaml` for
every content change because OCI chart versions are immutable.
- A merged `opencode-server` chart version is published to GHCR but does not
update the running server. A separate, explicitly approved
`kustomize-cluster` change must pin the `opencode` Argo CD Application to the
published version before GitOps can reconcile it.
- After `opencode-server` is published, the current workflow opens or updates a
`kustomize-cluster` pull request that pins the Argo CD Application to the new
version. The pull request still requires normal checks, review, and merge; it
does not sync Argo CD or deploy directly.
37 changes: 18 additions & 19 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,34 +1,33 @@
# charts

Helm chart monorepo for Make IT Work Cloud workloads. Each direct child
directory containing `Chart.yaml` is an independently validated and published
chart.
directory containing `Chart.yaml` is an independently versioned chart.

## Charts

| Chart | Purpose |
| --- | --- |
| `opencode-server/` | OpenCode server Deployment and its non-secret configuration ConfigMap |
| `opencode-server/` | OpenCode Deployment and non-secret configuration |

## Validation and publication
## Guides

Pull requests run repository hygiene plus `helm lint --strict`, `helm template`,
and package each changed chart without publishing it. A merge to `main` packages
changed charts and publishes them as immutable OCI artifacts to GHCR under:
- [Add a chart](docs/adding-a-chart.md)
- [Configure optional GitOps update automation](docs/gitops-update-automation.md)

## Release lifecycle

`Makefile` auto-discovers only direct children matching `<chart>/Chart.yaml`.
For both pull requests and `main`, CI runs repository hygiene and Helm validation
against every discovered chart. It then packages only charts changed by the
triggering commit. Pull requests stop there; a push to `main` publishes those
packages as immutable OCI artifacts under:

```text
oci://ghcr.io/makeitworkcloud/charts/<chart-name>
```

ArgoCD should consume a published chart by its explicit chart version. A future
full Helm migration may introduce an approved mechanism for selecting newer
chart versions automatically.

## Adding a chart

1. Add `<chart>/Chart.yaml` and templates.
2. Include chart-local configuration files under `<chart>/files/` when they are
rendered through Helm's `.Files` object.
3. Bump `version` in `Chart.yaml` for every published chart change; OCI chart
versions are immutable.
4. Update this README and open a PR.
Publication can optionally be followed by a pull request that updates a GitOps
consumer to the new version. That automation is currently implemented only for
`opencode-server`; the generated pull request changes desired state but does not
deploy or sync Argo CD. See the [GitOps automation guide](docs/gitops-update-automation.md)
for the current contract and extension requirements.
144 changes: 144 additions & 0 deletions docs/adding-a-chart.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
# Adding a chart

Add each chart as a direct child of this repository. The release workflow
discovers `<chart>/Chart.yaml`; deeper or grouped chart directories are not
included.

For the consuming GitOps resources, follow the
[kustomize-cluster workload guide](https://github.com/makeitworkcloud/kustomize-cluster/blob/main/docs/adding-a-workload.md).

## Create the chart

Use this structure:

```text
<chart>/
Chart.yaml
README.md
values.yaml
templates/
files/ # optional
```

An application chart should declare at least this metadata:

```yaml
apiVersion: v2
name: <chart-name>
description: <short purpose>
type: application
version: 0.1.0
appVersion: "<workload-version>"
```

- Keep the directory and chart `name` aligned. The metadata name becomes the
OCI package name.
- Use a valid semantic version for `version`. Bump it for every change to a
previously published chart, including templates and files; published OCI
versions are immutable.
- Set `appVersion` to the packaged workload version when one exists. Quote it
so YAML does not reinterpret the value.

## Preserve ownership boundaries

The chart should own portable workload resources such as Deployments and
non-secret application ConfigMaps. A workload-specific Service may also be
chart-owned when that is part of the agreed consumer contract.

By default, `kustomize-cluster` continues to own Secrets, Namespaces,
persistent storage, shared Services or networking, TunnelBindings, and
cluster/operator resources. Do not define the same resource in both
repositories. Document prerequisites and any intentional ownership migration
in the chart README and the consuming GitOps pull request.

## Organize configuration

- Put safe defaults and consumer overrides in `values.yaml`. Never put
credentials, tokens, kubeconfigs, or decrypted secret values there.
- Keep Kubernetes manifests in `templates/`. Use `_helpers.tpl` for repeated
naming and labels when useful, and ensure default values render successfully.
- Put packaged, non-secret static content in `files/` only when templates load
it through Helm's `.Files` API. A file change is a chart content change and
requires a new chart version.
- Add `<chart>/README.md` with the chart purpose, ownership boundary,
prerequisites, important values, rendered resources, and consumer example.
Keep operational and cluster rollout procedures in their canonical guides
rather than duplicating them.

Update the chart index in the root [README](../README.md) when adding or
removing a chart.

## Validate locally

Install Git, Helm, `jq`, pre-commit, and GNU `find` before using the repository
Make targets. Linux environments normally provide the required `find`
implementation. On macOS, put GNU findutils ahead of BSD `find` on `PATH`; if
that is not available, use the chart-specific Helm commands below and rely on
Linux CI for repository-wide discovery.

Run the same inexpensive checks used by CI when the tools are available:

```bash
pre-commit run --all-files
make test
helm lint --strict <chart>
helm template test <chart> >/dev/null
destination="$(mktemp -d)"
make package-chart CHART=<chart> DESTINATION="$destination"
```

`make test` lints and renders every discovered chart. The chart-specific
commands make failures easier to isolate, and packaging confirms the archive
can be built without publishing it.

## Open the pull request

Before requesting review, confirm:

- The chart is a direct child with complete `Chart.yaml` metadata and a chart
README.
- The chart version has never been published, or was bumped for this change.
- Rendered resource names, namespaces, values, and ownership match the intended
GitOps consumer.
- No secret or decrypted material is present.
- Local validation passed, or the pull request states which checks could not be
run.
- The root chart index and any affected consumer documentation are updated.

CI validates repository hygiene and every discovered chart on both pull
requests and `main`. Separately, `changed-charts` compares changed top-level
paths to the discovered chart list, so only changed charts are packaged. On a
pull request this proves packaging only; no OCI artifact is published.

## Publish and verify

After review and merge, the `main` workflow packages changed charts and pushes
them to GHCR. The package coordinates are:

```text
repository: ghcr.io/makeitworkcloud/charts
chart: <Chart.yaml name>
version: <Chart.yaml version>
OCI URI: oci://ghcr.io/makeitworkcloud/charts/<chart-name>
```

Confirm the `package <chart>` job succeeded, then inspect the published
metadata with authenticated GHCR access when required:

```bash
helm show chart oci://ghcr.io/makeitworkcloud/charts/<chart-name> \
--version <version>
```

Verify that the returned `name` and `version` match `Chart.yaml` before a
consumer pins the artifact.

If pull request validation or packaging fails, fix the cause and push another
commit. If a `main` publication fails before the artifact exists, correct the
transient or configuration failure and rerun the failed job. If publication
status is uncertain, check GHCR first: never change or republish content under
an existing version. Any content correction requires a new version.

Publication alone does not update a cluster. A chart may opt into the optional
[post-publish GitOps pull request](gitops-update-automation.md), but that
automation must be implemented explicitly for each chart.
111 changes: 111 additions & 0 deletions docs/gitops-update-automation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# Post-publish GitOps update automation

The release workflow can optionally open a pull request that pins a GitOps
consumer to a newly published chart version. This step is not generic today:
`.github/workflows/helm.yml` contains an `opencode-server`-specific job and
script. Adding another direct-child chart does not opt it into this automation.

The generated pull request changes Git desired state. It does not deploy,
merge, sync Argo CD, or prove rollout health.

## Current OpenCode contract

The `update-opencode-gitops` job runs only after a successful `main` package job
that included `opencode-server`. Its hard-coded chart-to-destination mapping is:

| Contract field | Current value |
| --- | --- |
| Chart directory and metadata name | `opencode-server` |
| Published OCI repository | `ghcr.io/makeitworkcloud/charts` |
| Destination | `makeitworkcloud/kustomize-cluster` (`main`) |
| Destination manifest | `workloads/apps/opencode-app.yaml` |
| Helm source | `opencode-server` in `ghcr.io/makeitworkcloud/charts` |
| Updated field | That source's single `targetRevision` |
| Automation branch | `automation/opencode-server-<version>` |

The script reads the version from `opencode-server/Chart.yaml`, requires
exactly one matching Helm source, updates only its `targetRevision`, commits the
change when needed, and creates a pull request if that branch has no open pull
request.

The job, condition, chart path, destination repository, manifest path, source
matcher, branch, commit, and pull request text are all OpenCode-specific
literals. They are not a reusable chart-to-consumer configuration model.

## Authentication and permissions

The job creates a short-lived GitHub App installation token scoped to the
destination repository. The App requires repository contents and pull request
write permissions; the workflow job itself otherwise retains read-only source
repository permissions.

The App private key is supplied through the charts repository Actions secret
`CHART_UPDATER_GITHUB_APP_PRIVATE_KEY`, which is managed by `tfroot-github`.
It is not retrieved from AWS. Never copy the key or secret payload. Keep the
non-secret App identifier in the canonical workflow configuration rather than
duplicating it in documentation or unrelated code, and read installation
identifiers from GitHub when needed. Follow the
[tfroot-github chart updater App runbook](https://github.com/makeitworkcloud/tfroot-github/blob/main/docs/chart-updater-github-app.md)
for installation scope, provisioning, and rotation.

## Branch and pull request behavior

- A version-specific branch is created from the destination's current `main`
only when the branch does not already exist.
- An existing branch is reused; the workflow does not rebase or reset it.
- The destination file must contain exactly one expected chart/repository
source. Zero or multiple matches fail without editing the file.
- The workflow commits only when `targetRevision` changes.
- An existing open pull request for the branch is reused. If none exists, the
workflow creates one with the version in its title and body.

These properties make a retry safe in the usual partial-failure cases, but a
stale or manually modified automation branch may require a reviewed cleanup
before retrying.

## Add another chart safely

1. Publish the chart normally and add its consuming Argo CD Application by
following the
[workload guide](https://github.com/makeitworkcloud/kustomize-cluster/blob/main/docs/adding-a-workload.md).
2. Define an explicit chart-to-destination contract: chart directory and name,
OCI repository, destination repository/base/path, exact source matcher,
target field, and branch/PR naming.
3. Add a dedicated job or refactor the workflow to use an explicit allowlisted
mapping. Update the changed-chart condition, `Chart.yaml` path, destination
literals, matcher, and branch/commit/PR text. Do not broaden the regex to
update arbitrary `targetRevision` fields.
4. Preserve the invariant that the updater runs only on `main`, only after that
chart was packaged successfully, and only after publication.
5. Keep the one-match assertion and least-privilege token scope. If the
destination is not already in the GitHub App installation and token request,
update `tfroot-github` through its runbook rather than adding another secret
source.
6. Link the chart README to the consumer and record whether GitOps updates are
automated or manual.

Before merge, validate workflow syntax and test the update logic against the
expected destination manifest or a fixture. Cover the intended one-match
update, zero matches, multiple matches, and an already-current version. The
pull request workflow can validate and package the changed chart, but the
post-publish job's `main` condition means its repository write cannot be tested
end to end from a pull request.

## Failure and retry

If chart packaging or publication fails, the updater does not run. Resolve the
release failure and verify the OCI artifact as described in
[Adding a chart](adding-a-chart.md).

If publication succeeded but the updater failed, the artifact remains valid
and nothing was deployed. Correct the token, permission, matcher, branch, or API
failure, then use the workflow's failed-job retry. A retry reuses an existing
version branch and open pull request. If the destination already pins the
version, verify whether a pull request is still necessary; creating a pull
request from a branch with no diff will fail.

After the pull request is created, normal `kustomize-cluster` checks, review,
and merge remain required. Follow the
[rollout and rollback guide](https://github.com/makeitworkcloud/kustomize-cluster/blob/main/docs/rollout-and-rollback.md)
to verify reconciliation and health or to return to a previous immutable
version. The generated pull request itself does not deploy.