Skip to content

[TASK] Give PIE the GitHub token of the run - #625

Merged
Kanti merged 5 commits into
webdevops:masterfrom
andersundsehr:ccr-57e46127-0as1io
Nov 16, 2026
Merged

Kanti merged 5 commits into
webdevops:masterfrom
andersundsehr:ccr-57e46127-0as1io

Conversation

@Kanti

@Kanti Kanti commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

PIE resolves and downloads the extensions through the GitHub API without a token, so its requests count against the limit for anonymous requests of the runner's IP address. Composer, on which PIE builds, reports every 401 or 403 of GitHub as "Could not authenticate" when it runs non-interactively, an exhausted limit too:

Could not download mongodb/mongodb-extension using pre-packaged-source method: Could not authenticate against github.com

In the week up to 2026-11-16 this failed 8 job attempts in 5 of 66 runs, 3 of which stayed red (nightly run, master after #623, master after #624), and 4 more jobs of the nightly run of 2026-11-16. All of them were PHP images on amd64.

Changes

  • PHP templates (32 Dockerfile.twig): the RUN instructions with PIE also mount --mount=type=secret,id=COMPOSER_AUTH,env=COMPOSER_AUTH. Composer, and so PIE, reads COMPOSER_AUTH. Without the secret the variable is empty and PIE works as before. Mounting a secret as an environment variable needs BuildKit 0.16.
  • GithubJobBuilder: Build (load locally), Build (load locally, from parent artifact) and Export image (OCI layout) pass COMPOSER_AUTH={"github-oauth":{"github.com":"${{ secrets.GITHUB_TOKEN }}"}} through secret-envs. Pull requests from forks get a read-only token too.
  • Permissions: the workflow had no permissions so far, so the token got the default of the repository settings, which is write access to everything (the "GITHUB_TOKEN Permissions" group at the start of a master job lists Contents: write, Packages: write, Actions: write, Issues: write and more). Of that, only Packages: write was used, for the existing pushes to ghcr.io. Now the workflow gives the token only contents: read; the image jobs get contents: read, packages: write, the publish jobs packages: write, because they push to ghcr.io.
  • Clean-up: the ghcr.io image of one architecture is named in one place (archImage()), which the image jobs and, with PLATFORMS, the publish jobs use; the login to ghcr.io is one helper for both. docker push now quotes the image name as a whole.
  • build-local.sh passes COMPOSER_AUTH to docker build when it is set.
  • Tests for the secret of every build step and for the permissions.
  • CONTRIBUTING.md: the CI section says what the token is for and what it may do; PHP extension builds names BuildKit 0.16 and shows how to build with your own token.

Trade-offs

  • Who can read the token: every script of the RUN instructions with PIE, not only PIE: the installers of Composer and Blackfire and the build systems of the extensions. In the image jobs on master the token may push to ghcr.io, and a token cannot have fewer permissions in a single step. It ends with the job. A read-only token for PIE would need the arch-image push to move out of the image jobs, or a separate token.
  • One limit for all runs: with the token, the PIE requests of all jobs and runs share the limit of GITHUB_TOKEN per repository instead of the anonymous limit of each runner's IP address (7 extensions in 16 PHP jobs and Xdebug in 48 development jobs per run).

Not changed

  • Docker Hub 429 Too Many Requests: in the same week 5 job attempts in 3 runs, all on one afternoon, 1 run stayed red. A mirror (mirror.gcr.io) was tried in this pull request and left out again: the failures are rare, and on master a mirror could hand an outdated official base image to the nightly rebuild.
  • Other infrastructure failures of the week (github.com downloads with 504, ghcr.io "unknown blob", keyserver) are left for a separate change.
  • [TASK] Add PHP 8.6 preview images on Debian trixie and Alpine #594 (PHP 8.6) adds Dockerfile.twig files with PIE. Whichever of the two is merged second adds the secret mount to them.

How it was verified

Change Check
PIE sends the token PIE 1.5.5 (ghcr.io/php/pie:bin) behind a local HTTPS proxy that logs the requests: pie download mongodb/mongodb-extension sends Authorization: token … to api.github.com/repos/mongodb/mongo-php-driver/releases/tags/2.5.7 with COMPOSER_AUTH set, and no Authorization header without it
PHP builds with the token Run of this pull request on b921d03 and the next one on 0476776: all jobs pass. In php:8.2-alpine (amd64) the step with PIE ran (not from the cache), found the prebuilt archive of mongodb through api.github.com/repos/mongodb/mongo-php-driver/releases/assets/… and installed all extensions; the log shows the secret as ***
Secret as environment variable docker build --secret id=COMPOSER_AUTH,env=COMPOSER_AUTH (Docker 29.9): the variable holds the secret, and set -x does not print it. Without --secret, or with the variable unset, the variable is empty and the build passes
BuildKit 0.16 moby/buildkit:v0.15.2 fails with unexpected key 'env' in 'env=COMPOSER_AUTH', v0.16.0 accepts it; the runners use v0.34.2
Permissions SonarCloud's finding githubactions:S8233 (write permission at workflow level) is gone since b921d03, and its quality gate passes
build-local.sh With a stub docker: --secret id=COMPOSER_AUTH,env=COMPOSER_AUTH with COMPOSER_AUTH set, no --secret without it
Generated files ci/console generate: compared with master, the workflow only gains the secret of the build steps and the permissions, and docker push quotes the image name as a whole; the 32 Dockerfiles gain the secret mount
ci/ PHPUnit: 277 tests pass locally; GrumPHP did not run locally, because phpstan/phpstan could not be downloaded there, so the job "Check ci/" runs it

Not verified yet: the job-level permissions on master (the push to ghcr.io with packages: write), which only the run after the merge shows.

…ror outside of master

Builds failed now and then on rate limits:

- PIE resolves and downloads the extensions through the GitHub API. It
  sent its requests without a token, so they counted against the limit
  for anonymous requests of the runner's IP address, and PIE failed with
  "Could not authenticate against github.com" (nightly run webdevops#450, pull
  request webdevops#624).
- BuildKit pulled the images from Docker Hub anonymously and got
  "429 Too Many Requests" (master run webdevops#452, pull requests webdevops#623 and webdevops#624).

The build steps now pass GITHUB_TOKEN as the build secret COMPOSER_AUTH,
which the RUN instructions with PIE mount as an environment variable for
Composer, on which PIE builds. Every script of those RUN instructions can
read the token, so the workflow restricts it to reading the repository
and writing the packages on ghcr.io. Mounting a secret as an environment
variable needs BuildKit 0.16.

Outside of master, BuildKit pulls the images from Docker Hub through
mirror.gcr.io; the webdevops images come from the artifacts of the run
there. Master still pulls from Docker Hub directly, because its builds
take the parent images that the publish jobs have just pushed.
…or there too

On master, the builds took their webdevops parent images from Docker
Hub, right after the publish jobs had pushed them. That is why they did
not go through mirror.gcr.io: the mirror could serve an outdated parent
image. They now take the image that the job of the parent pushed to
ghcr.io in the same run, through build-contexts, like the other runs take
it from the artifacts, and the login to ghcr.io comes before the build.

No build pulls a webdevops image from Docker Hub any more, so every run
pulls the images from Docker Hub through the mirror.
@Kanti Kanti changed the title [TASK] Give PIE a GitHub token and pull from Docker Hub through a mirror outside of master [TASK] Give PIE a GitHub token and pull from Docker Hub through a mirror Nov 15, 2026
The workflow gave packages: write to every job. Now the token may only
read the repository, and only the image jobs and the publish jobs, which
push to ghcr.io, may also write packages (SonarCloud githubactions:S8233).

The ghcr.io image of one architecture is written in one place now, which
also quotes the image name of "docker push" as a whole.
…ld-local.sh

The publish jobs wrote the ghcr.io images of amd64 and arm64 by hand;
they now take them from archImage() and PLATFORMS, like the image jobs.
The login to ghcr.io is a helper used by both jobs. The workflow does not
change.

build-local.sh passes COMPOSER_AUTH to docker build when it is set, so a
local build of all PHP images of a version can use a token for PIE too.

The comments say what the token of the run can do: every script of the
RUN instructions with PIE can read it, and in the image jobs it may push
to ghcr.io; all PIE requests share the limit of GITHUB_TOKEN.
…s from there again

In the week up to 2026-11-16, Docker Hub's "429 Too Many Requests" failed
5 jobs in 3 of 66 runs, all on one afternoon; PIE without a token failed
more often. A mirror would also let the nightly rebuild of master take an
outdated official base image, so the workflow no longer uses
mirror.gcr.io, and master takes its parent images from Docker Hub again,
as before this pull request. The login to ghcr.io goes back before the
push.

build-local.sh tests COMPOSER_AUTH with [[ ]] (SonarCloud shell:S7688),
and GithubJobBuilder is back to 19 methods (php:S1448).
@Kanti Kanti changed the title [TASK] Give PIE a GitHub token and pull from Docker Hub through a mirror [TASK] Give PIE the GitHub token of the run Nov 16, 2026
@sonarqubecloud

Copy link
Copy Markdown

@Kanti
Kanti marked this pull request as ready for review November 16, 2026 23:08
@Kanti
Kanti merged commit 9857399 into webdevops:master Nov 16, 2026
180 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant