Repository navigation
Give each package its own version and release only what changed - #63
Conversation
Every published module is a package in ci/packages.json, named by its artifactId and tagged <package>/<version>. A publish takes its plan from common-ci's steps/release-plan.ps1, which releases a package when a file under its folders changed since its last tag, and every package that depends on it, each at its last tag with the patch number moved. build-package.ps1 writes each module's own version into its pom.xml, with the versions of its parent and of the packages it depends on, and deploys only the planned modules. publish-package.ps1 tags and releases each one. check-version-bump.ps1 and check-release-interval.ps1 check each planned package against its own tags. build-project.ps1 fails a pull request whose poms do not match ci/packages.json. Step 3.5 of 51Degrees/common-ci#306.
common-ci's release plan calls ci/release-plan-filter.ps1 with the files under a package's folders that changed since its last tag, once its Ignore globs and the default globs for documentation, tests and examples have left theirs out. The filter leaves out a pom.xml whose only change is to its name, description, url, licenses, developers, scm or issueManagement, or to its comments and layout, so a new link does not release the package. A version, dependency, plugin or build change still counts, as does every other file. Its Pester tests are ci/release-plan-filter.Tests.ps1.
The example modules inside the published trees are packages of their own, but only the files under a folder named examples were ignored, so a change to the pom.xml, the web resources or the launcher of one released that module and every module depending on it. Each of the fourteen now ignores its whole folder. A release of a package an example depends on still releases the example, as before.
|
Pushed 8e0c2a3, "Release nothing for a change to an example module", to this branch. Examples, tests and documentation must never cause a release, and the default ignore list only covers files under a folder named ProblemFourteen published example modules sit outside the top-level What was checked
Change
How it was verifiedA temporary workflow ran common-ci
(Two earlier runs, 42686586924 and 42689442767, failed because my first test edits to a pom were comments, which BuildsCI configuration only (JSON and a comment), so James's waiver of the full build for text-only changes applies and no full build was run. RemainsNeeds common-ci#312 on |
Maven Central limits each organisation's release size and file count in a calendar month, so each publish now says how much it uploads.
A publish that stopped after its first tag push left the dependants untagged, and a run made again then planned nothing for them, because the release plan skips a package tagged at HEAD and finds dependants only from packages that changed. One atomic push leaves every package tagged or none.
…nks that are not 51Degrees pages
When gh release create fails part way, the tags are pushed and a publish run again plans nothing, so the missing GitHub releases are made by hand with the title and notes the script gives them.
Problem
Step 3.5 of the run book in common-ci#306.
ci/build-package.ps1line 31 ranmvn versions:setin each package tree, so every one of the 45 artifacts this repository publishes to Maven Central got a new version whenever anything changed. The last release, 4.5.130 on 5 November, published all 45 at that version.The run book's decisions are that each package has its own version, tagged
<package>/<version>and continuing from the version it last published, that a package is released when a file under its folders changed since its last tag together with every package that depends on it, and that only the patch number moves.The release #112 merged into
mainat 10:35 UTC on 13 November, and its Publish (run 43410313251) built all 45 at 4.5.131. Maven Central refused the upload with "Publishing blocked: your organization has reached its monthly release size limit. Grace period ended 2026-11-05. Usage resets 2026-12-01.", so nothing was published or tagged, and 4.5.130 is still the newest version of every package.Change
ci/packages.jsonlists the 45 published modules, each named by its artifactId, with its folder, the other folders its build takes files from, and the packages it depends on. A parent's folder is itspom.xmlalone, because the rest of its folder is its modules.DependsOnholds the parent and every dependency on another package in any scope, test jars included. The two on-premise engines andip-intelligence.translationtake files from thecxxsubmodule, andpipeline.didfrompipeline/owid-java, so a change to either submodule releases them.ci/release.ps1loads common-ci'ssteps/release-plan.ps1, which works out the plan, as 51degrees-python#47 does. It is kept apart fromci/packages.ps1, which every script loads, so building and testing a pull request does not need the shared step. It adds what Maven needs.Get-BuildPlangives the plan, or every package at its released version when nothing changed, so the nightly publish still has packages to test.Set-PomVersionswrites each published module's own version into itspom.xml, with the versions of its parent and of every package it depends on. A module that is not released stays at its last release, so a released module requires what is on Maven Central. The poms in the repository keep one development version.ci/build-package.ps1sets those versions and deploys only the planned modules, with-pl, so the one publishing bundle holds only them. It passes common-ci no folder to runversions:setin. The link it made attarget/central-publishingis gone, because common-ci'sjava/build-package.ps1has found the bundle under any module since common-ci#274.ci/build-package.ps1also writes the size and file count of the publishing bundle into the log, because Maven Central limits each organisation's release size and file count in a calendar month.ci/publish-package.ps1uploads the bundle, then pushes the tag of every planned package in one atomic push, makes a GitHub release for each, and tells common-ci not to add its own single tag. The release notes list the commits that touched the package's files. With one push for each tag, as this branch had before 9ad8a42, a publish that stopped after its first push left the dependants untagged, and a run made again then planned nothing for them, because the plan skips a package tagged at the commit and finds dependants only from packages that changed. That is the fault found in 51degrees-php#37 and 51degrees-node#37. One atomic push leaves every package tagged or none, as in 51degrees-go#60. Its Pester 5 tests areci/publish-package.Tests.ps1.ci/get-next-package-version.ps1gives common-ci the highest planned version as a label for the run, andci/package-update-required.ps1prints the plan and says whether there is anything to publish. Both are as in 51degrees-python.ci/check-version-bump.ps1andci/check-release-interval.ps1run common-ci's shared checks with-TagPrefixfor each planned package, so each is checked against its own tags for a major or minor move and for a release less than a week after its last. They are modelled on 51degrees-dotnet'sci/check-version-bump.ps1. common-ci'snightly-publish.ymlruns them, because it lists this repository inOWN_CHECK(run book step 2).ci/run-integration-tests.ps1andci/run-performance-tests.ps1point the examples at the version ofdevice-detection,ip-intelligenceandpipeline.jsonbuilderthe publish built, in place of the one version of the run.ci/build-project.ps1fails a pull request whose poms refer to a package itsDependsOndoes not name, or that adds a module with no entry, because a dependent left out would not be released with the package it depends on.ci/packages.ps1calls its list of trees$Trees, whichci/update-packages.ps1follows, so the word package means one thing. The rootpom.xml,README.md,AGENTS.mdandci/README.mddescribe the new scheme.No commit carries a version directive, and no major or minor number moves.
Checks at this head
The head is 48bedb3. Since the last body it has
rc(6e03a25) merged in at 713256e, and 48bedb3, which gives the filter's test fixtureshttps://example.comlinks in place of three untagged 51degrees.com links that the campaign tag lint failed (run 43533248603), and adds.github/workflows/ci-script-tests.ymlso the two Pester suites run on pull requests.rcnow has the same workflow, byte for byte, from #43.The head is now b2c9b9f6, which adds to
ci/README.mdhow to make by hand the GitHub releases that a publish stopped ingh release createleaves out (the review finding of 09:51 UTC on 14 November). It changes that one file and no script, so the dry run, the tests and the full build below, which ran on its parent 48bedb3, still hold. Under the decision of 11:35 UTC on 12 November for changes that touch only CI scripts and their notes ("rebuild and include, but the change doesn't impact the code so no need to run the full builds"), it needs only its quick checks, which are green, the CI script tests (run 43582975973, both Pester suites on Ubuntu and Windows), the environment variable names check (run 43582975969), the campaign tag lint (run 43582975999) and Only rc into main (run 43582976239) on b2c9b9f.A dry run of Publish (run 43559110241, passed). It ran common-ci's
nightly-publish.ymlatrc, withdryrunset, on a temporary branch holding 48bedb3 and two temporary commits, e4bf0a5 (the workflow that calls it) and 334a99c (setsPublishAnyBranchinci/install-package.ps1, so that the Package job runsci/publish-package.ps1offmain). The 45 first-release tags are on GitHub, so nothing else was needed.OWN_CHECK, so it ranci/check-version-bump.ps1andci/check-release-interval.ps1, which checked each of the 39 against its own tags. For each, the version check printed "Publishing 4.5.131, against 4.5.130 already released. The patch place, which needs no agreement.", and the weekly check printed a line such as "The last release, pipeline.core/4.5.130, was on 4 November 2026, so the next may go from 11 November 2026. This one is on 14 November 2026, so it may go." after noting that the weekly rule is not enforced whileWEEKLY_RELEASES_FROMis not set.pipeline.core,pipeline.did,pipeline.engines,pipeline.cloudrequestengine,pipeline.engines.fiftyone,pipeline.translation,pipeline.javascriptbuilder,pipeline.jsonbuilder,pipeline.web,pipeline.web.mvc,pipeline.geo-location,geo-location.cloud,device-detection.hash.engine.on-premise,device-detection.cloud,device-detection,ip-intelligence.shared,ip-intelligence.translation,ip-intelligence.engine.on-premise,ip-intelligence.cloudandip-intelligence. The other 19 depend on one of them. Almost every package depends onpipeline.core, whose changes this week include the configuration schema and the Pipeline builder.pipeline,pipeline.common,pipeline.caching,pipeline.web.packages,pipeline.developer-examplesandpipeline.developerexamples.web.shared.The atomic tag push, tests first. 8a2d77e5 adds
ci/publish-package.Tests.ps1, five Pester 5 tests that runci/publish-package.ps1with a stub upload script, a stub of common-ci's release plan, andgit,ghand the plan mocked, and 199df782 holds the change. Red, run 43531268900 ontmp/java-63-pub-red2(8a2d77e with a temporary workflow running the file withInvoke-Pesteronubuntu-latestandwindows-latest), failed "pushes every tag in one atomic push" with "Expected git to be called 1 times exactly, but was called 2 times" on both systems, and passed the other four (one release for each tag, no release when the push fails, nothing pushed in a dry run, nothing pushed for an empty plan). Green, run 43531267502 ontmp/java-63-pub-green2(199df78), passed all five on both.The CI script tests (run 43559011191, passed) ran
ci/publish-package.Tests.ps1andci/release-plan-filter.Tests.ps1on 48bedb3 on Ubuntu and Windows, and the campaign tag lint passed on the head (run 43559011208).The full pull request build (run 43559044228, passed) built 48bedb3 merged with
rc(6e03a25) on nine configurations, Ubuntu x64 with Java 8, 11, 17 and 21, Ubuntu arm64 with Java 17, and Windows with Java 8, 11, 17 and 21, and every job passed. Each of its tenSetting 'PullRequestSha'lines names 48bedb3, and its Configure job printed "Left out 6 macOS entries". macOS was left out of this draft's full build under the 9 November decision. The rc head is built with macOS before it merges into main.A trial merge with
rcat ceab088, which has since taken #27, #124 and #43, was clean on 14 November at 09:30 UTC. #27 changes onlydescriptionandscmin ten poms, whichci/release-plan-filter.ps1leaves out (cases 1 and 2 of the filter's dry run below).Checks on earlier heads
The checks ran on GitHub runners with common-ci at
rcchecked out at the root and this branch inside it, as common-ci's workflows lay them out. Each temporary branch was deleted when its run finished.The run book's checks 1 to 4 (run 42368761295, passed). The workflow tagged every package locally at this branch's head, made one local commit for each case, and ran the scripts the publish runs. Nothing was pushed.
ip-intelligence/ip-intelligence, plannedip-intelligence4.5.130 to 4.5.131 alone.DependsOnwith code of its own rather than the shared functions.pipeline/pipeline.coreplanned 34 packages,pipeline.corefirst andgeo-location.exampleslast.location/pom.xmlplanned the 9 packages of the location tree.cxxsubmodule planned the two on-premise engines andip-intelligence.translationas changed, anddevice-detection,ip-intelligenceandgeo-location.exampleswith them, 6 in all.ci/check-version-bump.ps1passed each package against its own tags.ci/options.json, a workflow, the rootpom.xml,AGENTS.md,pipeline/README.md, two Doxyfiles and three files underexamples/planned nothing.The same run also showed the following.
ci/package-update-required.ps1exited 1, and the build plan held all 45 packages at 4.5.130 for the tests.pipeline has never been released and packages.json gives it no FirstVersion, so a publish before the first-release tags exist releases nothing.pipeline.engines.fiftyone,pipeline.javascriptbuilderandpipeline.geo-locationchanged since 4.5.130, and the other 26 depend on them. The 16 left at 4.5.130 arepipeline,pipeline.common,pipeline.core,pipeline.did,pipeline.engines,pipeline.caching,pipeline.cloudrequestengine,pipeline.translation,pipeline.web.packages,pipeline.developer-examplesand six of its eight examples.Set-PomVersionswrote 4.5.131 into the 29 planned poms and 4.5.130 into the other 16, with each parent and dependency at its own package's version.pipeline.engines.fiftyone4.5.131, for example, requirespipeline.engines4.5.130, andpipeline.device-detection4.5.131 managespipeline-aggregate4.5.131.mvn -pl <the 29> validatethen read the rewritten reactor and the 29 modules at 4.5.131, BUILD SUCCESS.packages.jsoncheck found nothing missing as committed. Withpipeline.coretaken out of theDependsOnofpipeline.enginesand the entry forgeo-location.web.examples.sharedremoved, it named both.A dry run of the publish, through common-ci's
nightly-publish.ymlatrcwithdryrunset, with the first-release tags made in each job's checkout by a temporary step, and withPublishAnyBranchset so the Package job runsci/publish-package.ps1offmain(run 42369689241). The Configure job printed the same plan of 29 packages at 4.5.131 and the label 4.5.131. With this repository not yet in$OwnCheck, it ran the shared checks against the label. The version check passed, and the weekly check dated 4.5.130 by its GitHub release, 5 November, and reported that the next may go from 12 November, as a notice becauseWEEKLY_RELEASES_FROMis not set. The seven Pre Build jobs built the native libraries. The Build job deployed exactly the 29 planned modules, each at 4.5.131, signed them, and the publishing plugin wrote one bundle underpipeline/pipeline.engines.fiftyone/target/central-publishing, the first planned module, which common-ci'sjava/build-package.ps1found. The seven Test jobs, which install the built packages and test the examples againstdevice-detection,ip-intelligenceandpipeline.jsonbuilderat their built versions, and the Package job, which runsci/publish-package.ps1in dry run, all passed.The full pull request build, common-ci's
nightly-pull-request.ymlrun as a dry run from a temporary branch, built 70901e7, this branch's head, merged withrc(run 42423355619). All 15 build and test jobs passed, on Ubuntu x64 with Java 8, 11, 17 and 21, Ubuntu arm64 with Java 17, Windows with Java 8, 11, 17 and 21, and macOS arm64 and x64 with Java 11, 17 and 21. The first attempt failed only inMacOS_X64_Java_11, whereShareUsageTrackerTests.ShareUsageTracker_Session_DoNotTrackexpected 1 and got 2, the timing failure that #61 fixes, and the second attempt of that job passed.Documentation, tests, examples and pom metadata release nothing
James Rosewell decided on 7 November 2026 that a package is not released when the only changes under its folders since its last tag are to documentation, tests or examples, or only to metadata such as links in its manifest. common-ci#310 does that in
steps/release-plan.ps1. It leaves out Markdown,LICENSEfiles and files under folders nameddocs,tests,testorexamples, in any case, and then calls a repository's ownci/release-plan-filter.ps1, if it has one, with the files that are left.src/test. The default**/test/**matches a folder namedtestanywhere in the path, which takes in every module'ssrc/test, so no package needs anIgnorelist of its own. Case 4 below shows it.ci/release-plan-filter.ps1passes every file through except apom.xml. It reads thepom.xmlat the package's last tag and atHEAD, takes out the comments, the layout and thename,description,url,licenses,developers,scmandissueManagementchildren of<project>, and counts the file only when what is left differs. A version, dependency, plugin, build or repository change therefore still counts, and so does apom.xmlthat is new or removed.ci/release-plan-filter.Tests.ps1is a Pester 5 suite of 20 tests, each in a throwaway git repository with a taggedpom.xml..github/workflows/ci-script-tests.ymlruns it, withci/publish-package.Tests.ps1, on Ubuntu and Windows for each pull request that changesci/.ci/README.mdandREADME.mdsay which changes release nothing.rcmerged in four times, each with a merge commit. 2299507 brought Run the copyright update once a year #52, Run the Selenium contract tests against the on-premise web example too #54 and Run the IP Intelligence cloud tests again, and let a property have no value #57, and onlyci/run-integration-tests.ps1changed on both sides, without conflict. f4e129a brought 80 commits, andci/README.mdandci/build-package.ps1conflicted. Both keep this branch's text, becauserc's #78 had moved the version setting into common-ci withVersionPaths, which this branch already passes empty, and removed the same link this branch removed. 1abca1d brought Measure the share usage overhead as a median over warmed up rounds #91, and 713256e brought Test that a header no element reads reaches the usage sharing message #127, Write the JSON Builder's errors as a list of messages #128 and Name the builder defaults that were literals #130, both without conflict.Dry run
Run 42395465949 checked out common-ci at
feature/release-plan-ignore(2a31d28) at the root and this branch inside it, as common-ci's workflows lay them out, ran the filter's tests and then the plan throughci/release.ps1. Every tag and commit it made was local, and nothing was pushed.From the 4.5.130 first-release tags the plan is 29 packages at 4.5.131, as before, with the reasons now counting files, such as
pipeline.engines.fiftyone 4.5.130 4.5.131 changed since 4.5.130 (3 files). Each case below then started from every package tagged 4.5.131 at the branch head, which plans nothing, and added one commit.pipeline.corepom,descriptionandurlpipeline/pipeline.core/pom.xmlpipelineparent pom,developers,licensesandscmpipeline/pom.xmlpipeline.corepom, a comment and a blank linepipeline/pipeline.core/pom.xmlpipeline.core, a file undersrc/test.../src/test/java/.../OptionsTest.javapipeline.core, itsREADME.mdpipeline/pipeline.core/README.mdpipeline.corepom, the jar plugin'sAutomatic-Module-Namepipeline/pipeline.core/pom.xmlpipeline.core 4.5.131 4.5.132 changed since 4.5.131 (1 file)firstpipeline.corepom, a dependency version and thedescriptionpipeline/pipeline.core/pom.xmlpipeline.core, a source file undersrc/main.../src/main/java/.../BuildArg.javaci/release-plan-filter.ps1removedCase 10 shows that the filter, rather than the globs, is what leaves out case 1. The 34 are
pipeline.coreand every package that depends on it, as in the checks above.How the 7 November decision applies
James's decision names documentation, tests, examples and metadata, so each of the following releases nothing, as the decision says.
src/maintoo.location/geo-location.core/src/main/java/fiftyone/geolocation/core/test/MavenRunListener.javais in the publishedgeo-location.corejar but sits in atestfolder, and the main code ofgeo-location.examples,geo-location.web.examples.mvc,geo-location.web.examples.servletandgeo-location.web.examples.sharedsits inexamplesfolders. A change only there releases nothing, so those example packages are released only when a package they depend on is. The way to count them again isIgnoreDefaultsset tofalseon those entries with their ownIgnorelist. This branch leaves the defaults on, as the decision asks.pipeline.common,pipeline.core,pipeline.engines,device-detection.sharedandip-intelligence.sharedalso publish a test jar built fromsrc/test, and the examples use thedevice-detection.sharedone. A change only undersrc/testno longer releases them, so their published test jars stay as they were until something else changes.pom.xmldo not count. The decision named metadata such as links, and a comment changes neither the build nor what a project using the package gets.First release
Run book step 4. Every one of the 45 packages last published 4.5.130, on 5 November 2026, from
f69da52e, the commit of the plain tag4.5.130and the head ofmaintoday. Publish run 41934429106 built it from that commit, and Maven Central'smaven-metadata.xmlfor each of the 45 lists 4.5.130 as its newest 4.5 version, updated at 01:03 UTC on 5 November. So each tag is<artifactId>/4.5.130on that commit. The 45 tags were pushed on 7 November 2026, andgit ls-remote --tagsshows each of them onf69da52e.The tags are lightweight, as
ci/publish-package.ps1makes them, so the weekly check dates each by its commit, 23:31 UTC on 4 November. The plain4.5.130tag and its GitHub release stay as they are. The scripts onmainread only plain version tags, so the new tags can be pushed before this reachesmain.Maven Central
The 25 artifacts of the
pipelinetree show<release>4.6.0</release>in theirmaven-metadata.xml, from a minor version published on 18 September 2026 and withdrawn. Their newest 4.5 version is 4.5.130, and Central kept 4.6.0 as the release when 4.5.130 was published. The plan reads only this repository's tags, so 4.6.0 does not change it, and every package continues from 4.5.130.ci/update-packages.ps1takes patch updates only, so it does not move a requirement to 4.6.0 either. A project that asks Maven for the newest release of one of the 25, or gives a range that includes 4.6, still gets 4.6.0. Nothing here changes that.Maven Central's publishing limits say that once an organisation's grace period has ended, "new releases that would keep the organization over that metric's limit are no longer accepted", and that "Each month starts fresh". So nothing from this repository is published before 1 December, and the size of a release decides how much of a month it uses.
geo-location.web.examples.mvc(38.2 MB),pipeline.developerexamples.clientsideelement.mvc(38.2 MB) andgeo-location.web.examples.servlet(32.0 MB), because each war carries Tomcat and GraalJS (js-22.0.0.2.jaralone is 16 MB). The other 36 planned packages come to about 12 MiB.Order and what remains
steps/release-plan.ps1with the defaults and the filter, common-ci#307 and common-ci#310, reached common-ci'smainin common-ci#312 on 12 November, and itsnightly-publish.ymlthere lists 51degrees-java inOWN_CHECK, so the publish runs this repository's own checks for each package.mainafter this merges.rcstands.ci/README.mdsays. Agh release createthat fails part way leaves tags without their GitHub release, which a run made again does not make, so those are made by hand too, andci/README.mdgives the command, with the title and notes the script uses (b2c9b9f).device-detection/Dockerfile.examplesets the version fromgit describe --tags, which can now name a package tag. It is not used by CI and is left alone.pipeline.developer-exampleswith its eight examples, and the five location example modules, are published to Maven Central today, as they have been since 4.1, although their poms setmaven.deploy.skip, which the publishing plugin does not read. They stay packages here, so their streams continue. Whether they should go on being published is not decided here.