Repository navigation
setup_builder: set http/insecure only on the Docker mirror entry so buildx imagetools create can push Docker Hub manifest lists from the fallback builder (release/v2)
#151
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
piob-io
wants to merge
2
commits into
release/v2
Choose a base branch
from
devin/1795020504-fallback-dockerio-https-v2
base: release/v2
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+238
−8
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,96 @@ | ||
| #!/usr/bin/env bash | ||
| # Stands in for Docker Hub on this machine: registry-1.docker.io resolves to a | ||
| # local registry that serves TLS on :443 and, like the real Hub, answers every | ||
| # plain-HTTP request on :80 with a 301 to HTTPS. | ||
| # | ||
| # start serve a certificate from a CA added to the system trust store | ||
| # untrusted re-serve with a certificate from a CA nothing trusts | ||
| # http-log print the requests received on :80 | ||
| set -euo pipefail | ||
|
|
||
| DIR=/tmp/dockerhub-standin | ||
| HOST=registry-1.docker.io | ||
| HTTP_LOG="$DIR/http-requests.log" | ||
|
|
||
| make_cert() { | ||
| local name=$1 | ||
| openssl req -x509 -newkey rsa:2048 -nodes -days 1 -subj "/CN=$name CA" \ | ||
| -keyout "$DIR/$name-ca.key" -out "$DIR/$name-ca.crt" 2>/dev/null | ||
| openssl req -newkey rsa:2048 -nodes -subj "/CN=$HOST" \ | ||
| -keyout "$DIR/$name.key" -out "$DIR/$name.csr" 2>/dev/null | ||
| openssl x509 -req -in "$DIR/$name.csr" -days 1 \ | ||
| -CA "$DIR/$name-ca.crt" -CAkey "$DIR/$name-ca.key" -CAcreateserial \ | ||
| -extfile <(echo "subjectAltName=DNS:$HOST") -out "$DIR/$name.crt" 2>/dev/null | ||
| } | ||
|
|
||
| serve() { | ||
| local name=$1 | ||
| docker rm -f dockerhub-standin >/dev/null 2>&1 || true | ||
| docker run -d --name dockerhub-standin -p 127.0.0.1:443:443 \ | ||
| -v "$DIR:/certs:ro" \ | ||
| -e REGISTRY_HTTP_ADDR=0.0.0.0:443 \ | ||
| -e "REGISTRY_HTTP_TLS_CERTIFICATE=/certs/$name.crt" \ | ||
| -e "REGISTRY_HTTP_TLS_KEY=/certs/$name.key" \ | ||
| registry:2 >/dev/null | ||
| for _ in $(seq 30); do | ||
| if curl -fsS --cacert "$DIR/$name-ca.crt" "https://$HOST/v2/" >/dev/null 2>&1; then | ||
| return 0 | ||
| fi | ||
| sleep 1 | ||
| done | ||
| echo "stand-in registry did not come up" >&2 | ||
| docker logs dockerhub-standin >&2 | ||
| return 1 | ||
| } | ||
|
|
||
| case "${1:-}" in | ||
| start) | ||
| mkdir -p "$DIR" | ||
| # Pull while docker.io still resolves to the real Docker Hub. | ||
| docker pull -q registry:2 | ||
| make_cert trusted | ||
| make_cert untrusted | ||
| chmod a+r "$DIR"/*.key | ||
| sudo cp "$DIR/trusted-ca.crt" /usr/local/share/ca-certificates/dockerhub-standin.crt | ||
| sudo update-ca-certificates >/dev/null | ||
|
|
||
| cat >"$DIR/redirect.py" <<'PY' | ||
| import sys | ||
| from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer | ||
|
|
||
| LOG = sys.argv[1] | ||
|
|
||
|
|
||
| class Redirect(BaseHTTPRequestHandler): | ||
| def redirect(self): | ||
| with open(LOG, "a") as log: | ||
| log.write(f"{self.command} {self.path}\n") | ||
| self.send_response(301) | ||
| self.send_header("Location", f"https://{self.headers['Host']}{self.path}") | ||
| self.send_header("Content-Length", "0") | ||
| self.end_headers() | ||
|
|
||
| do_GET = do_HEAD = do_POST = do_PUT = do_PATCH = do_DELETE = redirect | ||
|
|
||
|
|
||
| ThreadingHTTPServer(("127.0.0.1", 80), Redirect).serve_forever() | ||
| PY | ||
| : >"$HTTP_LOG" | ||
| sudo nohup python3 "$DIR/redirect.py" "$HTTP_LOG" >"$DIR/redirect.out" 2>&1 & | ||
|
|
||
| echo "127.0.0.1 $HOST" | sudo tee -a /etc/hosts >/dev/null | ||
| serve trusted | ||
| curl -sS -o /dev/null -w '%{http_code}' "http://$HOST/v2/" | grep -qx 301 | ||
| : >"$HTTP_LOG" | ||
| ;; | ||
| untrusted) | ||
| serve untrusted | ||
| ;; | ||
| http-log) | ||
| cat "$HTTP_LOG" | ||
| ;; | ||
| *) | ||
| echo "usage: $0 start|untrusted|http-log" >&2 | ||
| exit 2 | ||
| ;; | ||
| esac |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| name: Test Docker Hub HTTPS | ||
|
|
||
| # Checks that buildx and buildkitd reach Docker Hub over verified HTTPS while | ||
| # the Blacksmith mirror stays on HTTP. .github/scripts/dockerhub-standin.sh | ||
| # answers as registry-1.docker.io on the runner. | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| pull_request: | ||
| push: | ||
| branches: [main, release/v2] | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| env: | ||
| IMAGE: docker.io/standin/setup-docker-builder-test | ||
|
|
||
| jobs: | ||
| fallback-builder: | ||
| name: Fallback builder pushes a manifest list over HTTPS | ||
| runs-on: blacksmith | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v6 | ||
|
|
||
| - name: Assert BLACKSMITH_AGENT_ADDR is advertised | ||
| run: test -n "$BLACKSMITH_AGENT_ADDR" | ||
|
|
||
| # No cache-key: the action warns and falls back to a local builder. | ||
| - name: Setup Docker Builder without cache-key | ||
| uses: ./ | ||
|
|
||
| - name: Assert the fallback builder is in use | ||
| run: | | ||
| docker buildx inspect --bootstrap | tee /tmp/builder.txt | ||
| grep -Eq '^Driver:\s+docker-container$' /tmp/builder.txt | ||
|
|
||
| - name: Start the Docker Hub stand-in | ||
| run: .github/scripts/dockerhub-standin.sh start | ||
|
|
||
| - name: Push a manifest list with imagetools create | ||
| run: | | ||
| docker buildx imagetools create -t "$IMAGE:manifest" registry.k8s.io/pause:3.10 | ||
| docker buildx imagetools inspect "$IMAGE:manifest" | ||
|
|
||
| - name: Assert nothing went to Docker Hub over plain HTTP | ||
| run: | | ||
| .github/scripts/dockerhub-standin.sh http-log | ||
| test ! -s /tmp/dockerhub-standin/http-requests.log | ||
|
|
||
| blacksmith-builder: | ||
| name: Blacksmith builder verifies Docker Hub TLS | ||
| runs-on: blacksmith | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v6 | ||
|
|
||
| - name: Assert BLACKSMITH_AGENT_ADDR is advertised | ||
| run: test -n "$BLACKSMITH_AGENT_ADDR" | ||
|
|
||
| # Before buildkitd starts, so it loads the stand-in's CA. | ||
| - name: Start the Docker Hub stand-in | ||
| run: .github/scripts/dockerhub-standin.sh start | ||
|
|
||
| - name: Setup Docker Builder | ||
| uses: ./ | ||
| with: | ||
| cache-key: test/setup-docker-builder-dockerhub-https | ||
| nofallback: "true" | ||
|
|
||
| - name: Assert the Blacksmith builder is in use | ||
| run: | | ||
| docker buildx inspect | tee /tmp/builder.txt | ||
| grep -Eq '^Driver:\s+remote$' /tmp/builder.txt | ||
|
|
||
| - name: Write a Dockerfile that pulls nothing | ||
| run: | | ||
| mkdir -p /tmp/ctx | ||
| date >/tmp/ctx/stamp | ||
| printf 'FROM scratch\nCOPY stamp /stamp\n' >/tmp/ctx/Dockerfile | ||
|
|
||
| - name: Push over HTTPS with a trusted certificate | ||
| run: docker buildx build --push -t "$IMAGE:trusted" /tmp/ctx | ||
|
|
||
| - name: Assert nothing went to Docker Hub over plain HTTP | ||
| run: | | ||
| .github/scripts/dockerhub-standin.sh http-log | ||
| test ! -s /tmp/dockerhub-standin/http-requests.log | ||
|
|
||
| - name: Serve a certificate no CA on the runner trusts | ||
| run: .github/scripts/dockerhub-standin.sh untrusted | ||
|
|
||
| - name: Assert the push fails on certificate verification | ||
| run: | | ||
| if docker buildx build --push -t "$IMAGE:untrusted" /tmp/ctx >/tmp/push.log 2>&1; then | ||
| cat /tmp/push.log | ||
| echo "::error::push succeeded against an untrusted certificate" | ||
| exit 1 | ||
| fi | ||
| cat /tmp/push.log | ||
| grep -q 'x509' /tmp/push.log | ||
| test ! -s /tmp/dockerhub-standin/http-requests.log | ||
Large diffs are not rendered by default.
Oops, something went wrong.
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.