Repository navigation
Add a DSpace lane with the BoringCache Maven tool cache - #92
Merged
Merged
Conversation
DSpace spends most of each build in one mvn package step that every source change invalidates, so no layer cache shortens it. The plus lane loads Maven build cache extension 1.3.0 and serves its remote cache through the BoringCache Docker Maven tool cache. Preparation writes overlay/dspace-maven.Dockerfile from each step's upstream Dockerfile: it adds .mvn/extensions.xml after the source copy and runs the assembly plugin's single goal even when the dspace module is restored, because the Dockerfile moves the dspace-installer directory that goal creates. It fails if either upstream line changes. There is no gha-plus lane. The build runs as the non-root dspace user, cache-dance restores cache mounts owned by root, and a uid or mode option gives the mount an identity cache-dance does not restore. rolling_series = 2 restarts every DSpace lane from start_sha with empty caches so the new lane has the same cold step 0. https://boringcache.com/docs/adapters/docker https://maven.apache.org/extensions/maven-build-cache-extension/parameters.html
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DSpace's warm builds are no faster than its cold build in any lane. On step 5, BoringCache reused 2 of 22 build steps;
RUN mvn packagetook 254 of the 309 seconds and reruns on every commit, because it follows the source copy. A layer cache can't skip it.This adds
boringcache-docker-plus(GitHub amd64, GitHub arm64 and local). It uses the layer cache plus the BoringCache Docker Maven tool cache, which serves the Maven build cache extension's remote cache inside that step.Changes
tools/docker/dspace/prepare.shwritesoverlay/dspace-maven.Dockerfilefrom each step's upstream Dockerfile. It changes two places:COPY --chown=dspace . /app/, it adds.mvn/extensions.xmlwithmaven-build-cache-extension1.3.0 (the CLI's default version). DSpace has no.mvn/directory, and the tool cache only sets the extension's properties.-Dmaven.build.cache.alwaysRunPlugins=maven-assembly-plugin:singletomvn package. The Dockerfile then movesdspace/target/dspace-installer, which that goal creates. Without the flag, a cache hit on thedspacemodule would skip the assembly and themvwould fail. The property exists in 1.3.0 and is documented in the extension parameters.If either upstream line changes, preparation fails instead of building a different Dockerfile. Base lanes still build the upstream Dockerfile unchanged.
tools/docker/dspace/plus/.boringcache.toml: the base plan plustool-cache = ["maven:dspace-maven"], building the overlay.tools/docker/dspace/case.toml: adds the lane, the prepare step androlling_series = 2.tools/README.md: documents the lane and the missing GitHub comparison lane.No
gha-pluslaneThe build runs as the non-root
dspaceuser. cache-dance restores a mount by first creating it as root, sodspacecan't write a persisted local build cache. Addinguidormodeto the mount gives it a different identity (BuildKit keys it with a base ref), so cache-dance would never restore it. The BoringCache lane doesn't have this problem, because its Maven scratch directory is writable by any user. The comparison is therefore BoringCache plus against the base lanes.Reseed
rolling_series = 2restarts every DSpace lane fromstart_shaon new*-rolling-2scopes, so the new lane gets the same cold step 0 as the others. Nushell used the same approach. Series-1 records stay inresults/and are no longer read. The repo rules say to delete series-1 BoringCache tags withboringcache rm --no-platform --no-git boringcache/benchmarks <tags>. I can't list or remove tags from my environment. By the plan's scoping they should bedspace-docker-boringcache-docker-github-rollinganddspace-docker-boringcache-docker-github-arm-rolling, but confirm against a listing first.Checks
bin/bench check: 15 tools, 45 cases: ok.Ruby 3.4.10 suite: 203 runs, 0 failures. Two
cli_testcases fail only under a US-ASCII locale; they pass under UTF-8, as on GitHub runners.bin/bench matrix dspace --tool Docker --rollingwith the current records plans all eight jobs at step 0, sha622fc02e1491, on-rolling-2scopes.The prepare script produces the two-hunk overlay against DSpace's Dockerfile, which is the same at
start_shaand today's head. It exits 1 on a Dockerfile without those lines.Fresh run of the new lane from this branch, run 43185821530, passed on both architectures:
In the cold amd64 build, Maven loaded the extension, found nothing in the remote cache, built every module and saved all 15 to BoringCache (
Saved to remote cache …/buildinfo.xml).mvn packagetook 4:09 min and the image check passed. The 15Cannot save project in cache, skippinglines come from the trailingmvn clean, which has nothing to save. The warm builds reused every layer, so Maven didn't run. A Maven cache restore will first happen at rolling step 1 after merge.