Repository navigation
[TASK] Give PIE the GitHub token of the run - #625
Merged
Merged
Conversation
…ror outside of master Builds failed now and then on rate limits: - PIE resolves and downloads the extensions through the GitHub API. It sent its requests without a token, so they counted against the limit for anonymous requests of the runner's IP address, and PIE failed with "Could not authenticate against github.com" (nightly run webdevops#450, pull request webdevops#624). - BuildKit pulled the images from Docker Hub anonymously and got "429 Too Many Requests" (master run webdevops#452, pull requests webdevops#623 and webdevops#624). The build steps now pass GITHUB_TOKEN as the build secret COMPOSER_AUTH, which the RUN instructions with PIE mount as an environment variable for Composer, on which PIE builds. Every script of those RUN instructions can read the token, so the workflow restricts it to reading the repository and writing the packages on ghcr.io. Mounting a secret as an environment variable needs BuildKit 0.16. Outside of master, BuildKit pulls the images from Docker Hub through mirror.gcr.io; the webdevops images come from the artifacts of the run there. Master still pulls from Docker Hub directly, because its builds take the parent images that the publish jobs have just pushed.
…or there too On master, the builds took their webdevops parent images from Docker Hub, right after the publish jobs had pushed them. That is why they did not go through mirror.gcr.io: the mirror could serve an outdated parent image. They now take the image that the job of the parent pushed to ghcr.io in the same run, through build-contexts, like the other runs take it from the artifacts, and the login to ghcr.io comes before the build. No build pulls a webdevops image from Docker Hub any more, so every run pulls the images from Docker Hub through the mirror.
The workflow gave packages: write to every job. Now the token may only read the repository, and only the image jobs and the publish jobs, which push to ghcr.io, may also write packages (SonarCloud githubactions:S8233). The ghcr.io image of one architecture is written in one place now, which also quotes the image name of "docker push" as a whole.
…ld-local.sh The publish jobs wrote the ghcr.io images of amd64 and arm64 by hand; they now take them from archImage() and PLATFORMS, like the image jobs. The login to ghcr.io is a helper used by both jobs. The workflow does not change. build-local.sh passes COMPOSER_AUTH to docker build when it is set, so a local build of all PHP images of a version can use a token for PIE too. The comments say what the token of the run can do: every script of the RUN instructions with PIE can read it, and in the image jobs it may push to ghcr.io; all PIE requests share the limit of GITHUB_TOKEN.
…s from there again In the week up to 2026-11-16, Docker Hub's "429 Too Many Requests" failed 5 jobs in 3 of 66 runs, all on one afternoon; PIE without a token failed more often. A mirror would also let the nightly rebuild of master take an outdated official base image, so the workflow no longer uses mirror.gcr.io, and master takes its parent images from Docker Hub again, as before this pull request. The login to ghcr.io goes back before the push. build-local.sh tests COMPOSER_AUTH with [[ ]] (SonarCloud shell:S7688), and GithubJobBuilder is back to 19 methods (php:S1448).
|
Kanti
marked this pull request as ready for review
November 16, 2026 23:08
This was referenced Nov 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Summary
PIE resolves and downloads the extensions through the GitHub API without a token, so its requests count against the limit for anonymous requests of the runner's IP address. Composer, on which PIE builds, reports every 401 or 403 of GitHub as "Could not authenticate" when it runs non-interactively, an exhausted limit too:
In the week up to 2026-11-16 this failed 8 job attempts in 5 of 66 runs, 3 of which stayed red (nightly run, master after #623, master after #624), and 4 more jobs of the nightly run of 2026-11-16. All of them were PHP images on amd64.
Changes
Dockerfile.twig): theRUNinstructions with PIE also mount--mount=type=secret,id=COMPOSER_AUTH,env=COMPOSER_AUTH. Composer, and so PIE, readsCOMPOSER_AUTH. Without the secret the variable is empty and PIE works as before. Mounting a secret as an environment variable needs BuildKit 0.16.GithubJobBuilder:Build (load locally),Build (load locally, from parent artifact)andExport image (OCI layout)passCOMPOSER_AUTH={"github-oauth":{"github.com":"${{ secrets.GITHUB_TOKEN }}"}}throughsecret-envs. Pull requests from forks get a read-only token too.permissionsso far, so the token got the default of the repository settings, which is write access to everything (the "GITHUB_TOKEN Permissions" group at the start of a master job listsContents: write,Packages: write,Actions: write,Issues: writeand more). Of that, onlyPackages: writewas used, for the existing pushes to ghcr.io. Now the workflow gives the token onlycontents: read; the image jobs getcontents: read, packages: write, the publish jobspackages: write, because they push to ghcr.io.archImage()), which the image jobs and, withPLATFORMS, the publish jobs use; the login to ghcr.io is one helper for both.docker pushnow quotes the image name as a whole.build-local.shpassesCOMPOSER_AUTHtodocker buildwhen it is set.CONTRIBUTING.md: the CI section says what the token is for and what it may do; PHP extension builds names BuildKit 0.16 and shows how to build with your own token.Trade-offs
RUNinstructions with PIE, not only PIE: the installers of Composer and Blackfire and the build systems of the extensions. In the image jobs on master the token may push to ghcr.io, and a token cannot have fewer permissions in a single step. It ends with the job. A read-only token for PIE would need the arch-image push to move out of the image jobs, or a separate token.GITHUB_TOKENper repository instead of the anonymous limit of each runner's IP address (7 extensions in 16 PHP jobs and Xdebug in 48 development jobs per run).Not changed
429 Too Many Requests: in the same week 5 job attempts in 3 runs, all on one afternoon, 1 run stayed red. A mirror (mirror.gcr.io) was tried in this pull request and left out again: the failures are rare, and on master a mirror could hand an outdated official base image to the nightly rebuild.Dockerfile.twigfiles with PIE. Whichever of the two is merged second adds the secret mount to them.How it was verified
ghcr.io/php/pie:bin) behind a local HTTPS proxy that logs the requests:pie download mongodb/mongodb-extensionsendsAuthorization: token …toapi.github.com/repos/mongodb/mongo-php-driver/releases/tags/2.5.7withCOMPOSER_AUTHset, and noAuthorizationheader without itb921d03and the next one on0476776: all jobs pass. Inphp:8.2-alpine (amd64)the step with PIE ran (not from the cache), found the prebuilt archive of mongodb throughapi.github.com/repos/mongodb/mongo-php-driver/releases/assets/…and installed all extensions; the log shows the secret as***docker build --secret id=COMPOSER_AUTH,env=COMPOSER_AUTH(Docker 29.9): the variable holds the secret, andset -xdoes not print it. Without--secret, or with the variable unset, the variable is empty and the build passesmoby/buildkit:v0.15.2fails withunexpected key 'env' in 'env=COMPOSER_AUTH',v0.16.0accepts it; the runners use v0.34.2githubactions:S8233(write permission at workflow level) is gone sinceb921d03, and its quality gate passesbuild-local.shdocker:--secret id=COMPOSER_AUTH,env=COMPOSER_AUTHwithCOMPOSER_AUTHset, no--secretwithout itci/console generate: compared with master, the workflow only gains the secret of the build steps and the permissions, anddocker pushquotes the image name as a whole; the 32 Dockerfiles gain the secret mountci/phpstan/phpstancould not be downloaded there, so the job "Check ci/" runs itNot verified yet: the job-level permissions on master (the push to ghcr.io with
packages: write), which only the run after the merge shows.