Skip to content

Make the build reproducible by deriving version metadata from the manifest - #489

Open
ppkarwasz wants to merge 2 commits into
trunkfrom
reproducible-version-info
Open

Make the build reproducible by deriving version metadata from the manifest#489
ppkarwasz wants to merge 2 commits into
trunkfrom
reproducible-version-info

Conversation

@ppkarwasz

Copy link
Copy Markdown
Member

VersionInfo was the last part of the build that could not produce the same output twice.

saveVersion.sh (and its PowerShell twin) generated a @VersionAnnotation at generate-sources recording the builder's user name, host name, working directory, wall clock and an MD5 of every *.java file in the tree. Six of the seven fields were therefore machine-specific, and the source distribution generated by logging-parent carries no repository metadata at all, so a build from it could never match a build from a Git checkout.

The same facts now come from manifest headers derived from the POM, so a Git checkout and the source archive produce byte-identical artifacts.

Manifest

Three headers are appended to the BND-generated manifest through bnd-extra-config, which puts them in target/classes as well as the JAR, so the values are also available to tests and in the IDE:

Header Source Specified by
Bundle-SCM <scm> OSGi Core R8, section 3.2.1
Implementation-Timestamp project.build.outputTimestamp nothing, see below
Purl groupId, artifactId, version the value follows the package-url specification, the header name is ours

Neither the JAR File Specification nor OSGi defines a header for a build timestamp or a package identifier, so those two are Flume-specific. Implementation-Vendor-Id was considered for the latter and rejected: it is deprecated in java.util.jar.Attributes.Name as a leftover of the extension mechanism removed by JEP 220.

VersionInfo

Method Source
getVersion() Implementation-Version, then pom.properties
getPurl() (new) Purl, then built from pom.properties
getUrl() Bundle-SCM attribute url
getBranch() Bundle-SCM attribute tag
getDate() Implementation-Timestamp
getRevision() fixed, until the build records a commit id again
getUser(), getSrcChecksum() fixed, deprecated for removal

The manifest is resolved against the location of VersionInfo.class rather than looked up on the class path, so another artifact cannot answer, and it is accepted only when its Purl identifies flume-ng-core. When Flume has been shaded into an uber JAR the manifest belongs to the shading project, so it is rejected and META-INF/maven/org.apache.flume/flume-ng-core/pom.properties is used instead.

VersionAnnotation, both saveVersion scripts and the two OS-conditional profiles that ran them are removed.

Also fixed

flume-parent, which is the parent of every published module, never received the revision and project.build.outputTimestamp conventions applied to the aggregator in #467. It still carried a 2022 timestamp and a hard-coded version, and since the release automation patches only the root POM, neither would have been updated at release time. The first commit applies the convention there and switches all child modules to ${revision}.

Verification

  • Two clean builds of flume-ng-core produce the same SHA-256.
  • flume-ng version prints identical output from the JAR and from target/classes.
  • The shaded fallback is exercised against a real uber JAR built with a foreign manifest.
  • mvn verify is green; PMD and SpotBugs counts are unchanged.

Still open

The release workflow in logging-parent bumps revision and project.build.outputTimestamp in the root POM only, so flume-parent will go stale at release time. That needs a fix upstream.

Every published module inherits from `flume-parent`, not from the
aggregator, so the CI-friendly properties have to be declared there as
well. Without them the release automation bumps only the aggregator and
the artifacts ship a stale version and build timestamp.

Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011QwMh1JvFaPBgWEGrMgMj7
`saveVersion.sh` recorded the builder's user name, host, clock and a
checksum of the working copy, so no two builds agreed and the source
distribution, which carries no repository metadata, agreed with none.
The same facts now come from POM-derived manifest headers, which a Git
checkout and the source archive produce identically.

`getUser()` and `getSrcChecksum()` are deprecated for removal, and
`getRevision()` reports nothing until the build records a commit id
again.

Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011QwMh1JvFaPBgWEGrMgMj7
@ppkarwasz
ppkarwasz requested a review from rgoers August 21, 2026 10:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant