Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/contributing/howto/add-an-action.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@ nix-setup/action.yaml → genlayerlabs/github-actions/nix-setup@main
empty, not just what it is.
- Never `set -x` in a step that handles a token.
- An action that talks to the cache must not fail the job when the cache is
unavailable. Warn and continue.
unavailable. Warn and continue — unless the caller opts in to failing, as
`nix-cache-push` does with `verify: "true"`. The default stays warn-only, so
no existing consumer changes behaviour.

## Testing

Expand Down
26 changes: 17 additions & 9 deletions docs/using/actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,13 +57,14 @@ Notes:

Uploads paths this job built locally.

| Input | Default | Description |
| ---------- | -------------------------- | -------------------------------------------- |
| `token` | `""` | push token; empty skips the step entirely |
| `cache` | `genlayer` | cache name on the server |
| `endpoint` | `https://nix-cache.ygr.ai` | attic server |
| `include` | `genlayer\|genvm` | regex on the store path name, hash stripped |
| `exclude` | `""` | regex on the full store path; empty disables |
| Input | Default | Description |
| ---------- | -------------------------- | -------------------------------------------------------------------------- |
| `token` | `""` | push token; empty skips the step entirely |
| `cache` | `genlayer` | cache name on the server |
| `endpoint` | `https://nix-cache.ygr.ai` | attic server |
| `include` | `genlayer\|genvm` | regex on the store path name, hash stripped |
| `exclude` | `""` | regex on the full store path; empty disables |
| `verify` | `"false"` | `"true"` fails the job unless every pushed path is in the cache afterwards |

Notes:

Expand All @@ -73,7 +74,14 @@ Notes:
- Guard the step with `if: ${{ !cancelled() }}`, not the default `success()`. A
job that failed late still built most of the closure, and that is the work a
retry should not repeat.
- Every failure warns and exits 0. A cache that refuses uploads must not turn a
green build red.
- By default every failure warns and exits 0. A cache that refuses uploads must
not turn a green build red.
- `verify: "true"` is the opt-in exception, for a job whose output consumers
depend on (a release, the build that warms a branch head). A failed client
build, path listing, login or upload exits 1, and each pushed path is then
read back (`<endpoint>/<cache>/<hash>.narinfo` with the push token, which can
pull): any answer but 200 fails the job and names the path and the HTTP code.
This is what makes an expired push token visible — without it a refused push
is only a warning in a green job. It needs `curl`.
- Requires `jq` and a flake in `$GITHUB_WORKSPACE` (`--inputs-from .` resolves
the attic client through your lock file rather than the mutable registry).
65 changes: 57 additions & 8 deletions nix-cache-push/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,17 @@ inputs:
description: regex matched against the full store path; empty disables
required: false
default: ""
verify:
description: >-
"true" fails the job unless every pushed path is in the cache afterwards:
a failed client build, path listing, login or upload exits 1 instead of
warning, and each pushed path is then read back from the cache with the
push token. Set it on a job whose output consumers depend on (a release,
the build that warms a branch head); an expired token then shows as a red
job instead of a warning. Default "false": a cache that refuses uploads
never turns a green build red. An empty token still skips the step.
required: false
default: "false"
runs:
using: composite
steps:
Expand All @@ -40,19 +51,30 @@ runs:
ENDPOINT: ${{ inputs.endpoint }}
INCLUDE: ${{ inputs.include }}
EXCLUDE: ${{ inputs.exclude }}
VERIFY: ${{ inputs.verify }}
run: |
cd "$GITHUB_WORKSPACE"

# A cache that refuses uploads must not turn a green build red, so every
# step of this script warns and gives up instead of failing the job.
# step of this script warns and gives up instead of failing the job —
# unless the caller asked for `verify`, where a missing upload is the
# failure the job exists to report.
give_up() {
if [ "$VERIFY" = "true" ]; then
echo "::error::$1"
exit 1
fi
echo "::warning::$1"
exit 0
}

attic_link="${RUNNER_TEMP:-/tmp}/attic-client"
# An out-link rather than `nix run`: it is a gcroot, and a second call
# in the same job skips both evaluation and substitution.
# `--inputs-from .` resolves `nixpkgs` through our flake lock, so the
# client comes from the pinned revision instead of the mutable registry.
if ! nix build --out-link "$attic_link" --inputs-from . nixpkgs#attic-client; then
echo "::warning::could not build the attic client; nothing pushed"
exit 0
give_up "could not build the attic client; nothing pushed"
fi

# `ultimate` is nix's own flag for "built on this machine", which is
Expand All @@ -67,7 +89,10 @@ runs:
| select($inc == "" or (.key | sub("^/nix/store/[^-]+-"; "") | test($inc)))
| select(.key | test("(-|\\.)(source|socket|service|env|pam|drv)$") | not)
| select($exc == "" or (.key | test($exc) | not))
| .key' > "$paths" || echo "::error::failed to enumerate locally-built paths"
| .key' > "$paths" || {
[ "$VERIFY" != "true" ] || give_up "failed to enumerate locally-built paths"
echo "::error::failed to enumerate locally-built paths"
}

echo "::group::artifacts"
cat "$paths"
Expand All @@ -80,12 +105,36 @@ runs:
echo "pushing $(wc -l < "$paths") path(s) to $CACHE"

if ! "$attic_link/bin/attic" login ci "$ENDPOINT" "$NIX_CACHE_PUSH_TOKEN" --set-default; then
echo "::warning::attic login failed; nothing pushed"
exit 0
give_up "attic login failed; nothing pushed"
fi

# Redirect stdin from the path list rather than `xargs -a FILE`: -a is a
# GNU extension and BSD/macOS xargs rejects it, which broke the push on
# macOS runners. `< file` reads the same list and is portable.
< "$paths" xargs "$attic_link/bin/attic" push --jobs 4 "$CACHE" \
|| echo "::warning::attic push failed"
if ! < "$paths" xargs "$attic_link/bin/attic" push --jobs 4 "$CACHE"; then
give_up "attic push failed (Unauthorized means the push token expired)"
fi

[ "$VERIFY" = "true" ] || exit 0

# Read every pushed path back from the cache, so a push that reported
# success but left a path out still fails. The push token can pull, and
# it reaches curl through a config on stdin, not the command line.
checked=0
missing=0
while IFS= read -r path; do
checked=$((checked + 1))
hash="$(basename "$path")"
hash="${hash%%-*}"
code="$(printf 'header = "Authorization: Bearer %s"\n' "$NIX_CACHE_PUSH_TOKEN" \
| curl -sS -o /dev/null -w '%{http_code}' -K - "$ENDPOINT/$CACHE/$hash.narinfo" || true)"
if [ "$code" != "200" ]; then
echo "::error::$path is not in the $CACHE cache (HTTP $code)"
missing=$((missing + 1))
fi
done < "$paths"
if [ "$missing" -gt 0 ]; then
echo "::error::$missing pushed path(s) are not in the $CACHE cache"
exit 1
fi
echo "verified $checked path(s) in $CACHE"
Loading