Make the build reproducible by deriving version metadata from the manifest - #489
Open
ppkarwasz wants to merge 2 commits into
Open
Make the build reproducible by deriving version metadata from the manifest#489ppkarwasz wants to merge 2 commits into
ppkarwasz wants to merge 2 commits into
Conversation
Every published module inherits from `flume-parent`, not from the aggregator, so the CI-friendly properties have to be declared there as well. Without them the release automation bumps only the aggregator and the artifacts ship a stale version and build timestamp. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011QwMh1JvFaPBgWEGrMgMj7
`saveVersion.sh` recorded the builder's user name, host, clock and a checksum of the working copy, so no two builds agreed and the source distribution, which carries no repository metadata, agreed with none. The same facts now come from POM-derived manifest headers, which a Git checkout and the source archive produce identically. `getUser()` and `getSrcChecksum()` are deprecated for removal, and `getRevision()` reports nothing until the build records a commit id again. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011QwMh1JvFaPBgWEGrMgMj7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
VersionInfowas the last part of the build that could not produce the same output twice.saveVersion.sh(and its PowerShell twin) generated a@VersionAnnotationatgenerate-sourcesrecording the builder's user name, host name, working directory, wall clock and an MD5 of every*.javafile in the tree. Six of the seven fields were therefore machine-specific, and the source distribution generated bylogging-parentcarries no repository metadata at all, so a build from it could never match a build from a Git checkout.The same facts now come from manifest headers derived from the POM, so a Git checkout and the source archive produce byte-identical artifacts.
Manifest
Three headers are appended to the BND-generated manifest through
bnd-extra-config, which puts them intarget/classesas well as the JAR, so the values are also available to tests and in the IDE:Bundle-SCM<scm>Implementation-Timestampproject.build.outputTimestampPurlgroupId,artifactId,versionNeither the JAR File Specification nor OSGi defines a header for a build timestamp or a package identifier, so those two are Flume-specific.
Implementation-Vendor-Idwas considered for the latter and rejected: it is deprecated injava.util.jar.Attributes.Nameas a leftover of the extension mechanism removed by JEP 220.VersionInfogetVersion()Implementation-Version, thenpom.propertiesgetPurl()(new)Purl, then built frompom.propertiesgetUrl()Bundle-SCMattributeurlgetBranch()Bundle-SCMattributetaggetDate()Implementation-TimestampgetRevision()getUser(),getSrcChecksum()The manifest is resolved against the location of
VersionInfo.classrather than looked up on the class path, so another artifact cannot answer, and it is accepted only when itsPurlidentifiesflume-ng-core. When Flume has been shaded into an uber JAR the manifest belongs to the shading project, so it is rejected andMETA-INF/maven/org.apache.flume/flume-ng-core/pom.propertiesis used instead.VersionAnnotation, bothsaveVersionscripts and the two OS-conditional profiles that ran them are removed.Also fixed
flume-parent, which is the parent of every published module, never received therevisionandproject.build.outputTimestampconventions applied to the aggregator in #467. It still carried a 2022 timestamp and a hard-coded version, and since the release automation patches only the root POM, neither would have been updated at release time. The first commit applies the convention there and switches all child modules to${revision}.Verification
flume-ng-coreproduce the same SHA-256.flume-ng versionprints identical output from the JAR and fromtarget/classes.mvn verifyis green; PMD and SpotBugs counts are unchanged.Still open
The release workflow in
logging-parentbumpsrevisionandproject.build.outputTimestampin the root POM only, soflume-parentwill go stale at release time. That needs a fix upstream.