Skip to content

[TASK] Pass the image test runner on to the image jobs as an artifact - #633

Draft
Kanti wants to merge 1 commit into
webdevops:masterfrom
andersundsehr:c/laughing-carson-ppsduw
Draft

Kanti wants to merge 1 commit into
webdevops:masterfrom
andersundsehr:c/laughing-carson-ppsduw

Conversation

@Kanti

@Kanti Kanti commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Every image job built ci/image-tests/Dockerfile itself, with cache-from the cache that the job image-test-runner writes. Even when every step comes from the cache, BuildKit resolves the metadata of php:8.4-cli-alpine and composer:2 on Docker Hub, anonymously: about 300 requests per run. On 2026-11-17 two jobs failed in that step with failed to resolve source metadata for docker.io/library/composer:2: ... 429 Too Many Requests:

Change

  • The job image-test-runner builds the runner as before, with the cache of GitHub Actions, but exports it as an archive for docker load (outputs: type=docker,dest=.ci-image-test-runner/webdevops-image-tests.tar) and uploads it as the artifact webdevops-image-tests-<arch>: kept for one day, replaced on a rerun, not compressed again (its layers are gzip-compressed already).
  • The image jobs download that artifact and run docker load --input .ci-image-test-runner/webdevops-image-tests.tar, without a request to Docker Hub.
  • The fallback stays: download and load have continue-on-error, and only if one of them fails, e.g. because image-test-runner could not build or upload the runner, the step "Build the image test runner" runs as before, from the cache. The build and upload steps of image-test-runner keep continue-on-error, so a failure there does not skip the image jobs.
  • Not ghcr.io: pull requests from forks get a read-only GITHUB_TOKEN, so the runner job could not push the runner there. Artifacts work for them, as they do for the OCI layouts of the parent images.
  • Size: the artifact is about 133 MB per architecture (134.1 MB for amd64, 132.8 MB for arm64 in run 44128843023). Each image job with image tests downloads it once.

Documentation

  • CONTRIBUTING.md, section "Image tests": the job image-test-runner passes the runner on as an artifact, the image jobs load it with docker load, and if that fails they build it themselves.
  • AGENTS.md and the comment in the Makefile: the CI loads the runner instead of building it.

Verification

  • ci/console generate (the command of make generate): only .github/workflows/build.yaml changes.
  • GrumPHP (the command of make check): composer, phpcs, PHPStan, Rector, Fractor, composer audit and PHPUnit pass. New tests check that the runner job exports and uploads the archive, that the image jobs download and load it and build the runner only if that fails, and that every steps.<id> in an if refers to an earlier step of the same job. They fail against the old GithubJobBuilder.
  • Locally: docker buildx build --tag webdevops-image-tests --output type=docker,dest=<missing dir>/webdevops-image-tests.tar creates the directory and names the image after the tag, with the docker and the docker-container driver. docker load of the archive gives webdevops-image-tests:latest, and make test-image-run IMAGE=webdevops/php:8.4-alpine with the loaded runner passes (137 tests, 5 skipped).
  • CI, run 44128843023 of this pull request: both jobs "Image test runner" uploaded the artifact. In every image job with image tests of the run (114 jobs), "Download the image test runner" and "Load the image test runner" succeeded, "Build the image test runner" was skipped and the image tests passed. The logs of 7 of them (base:alpine, php:8.4 and php:8.4-alpine on both architectures, php-nginx:8.3 on arm64) show Loaded image: webdevops-image-tests:latest and no load metadata for docker.io/library/php or docker.io/library/composer; only the two runner jobs still ask Docker Hub for them.

Every image job built ci/image-tests/Dockerfile from the cache of the
job image-test-runner. Even with every step cached, BuildKit asks
Docker Hub for the metadata of php:8.4-cli-alpine and composer:2, so
each run sent about 300 anonymous requests to Docker Hub, and jobs
failed with "429 Too Many Requests" in that step.

The job image-test-runner now exports the runner per architecture as
an archive for docker load (type=docker) and uploads it as the
artifact webdevops-image-tests-<arch>, still with the cache of GitHub
Actions. The image jobs download it and load it with docker load,
without a request to Docker Hub. Only if the download or the load
fails, e.g. because that job could not build the runner, they build
it themselves from the cache as before.

CONTRIBUTING.md, AGENTS.md and the comment in the Makefile describe
the new way.
@Kanti
Kanti force-pushed the c/laughing-carson-ppsduw branch from 47093a2 to 221011e Compare November 17, 2026 16:42
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant