From 83d931d699cf13d3c716fc8e7d1337573025df87 Mon Sep 17 00:00:00 2001 From: GeiserX <9169332+GeiserX@users.noreply.github.com> Date: Sun, 15 Nov 2026 18:56:42 +0100 Subject: [PATCH 1/2] build(deps): hold ExpansionHunter at its Apache-2.0 release Bead: lic-32 --- CHANGELOG.md | 2 ++ docs/updating.md | 2 ++ docs/versions.md | 2 +- renovate.json | 6 ++++++ versions.env | 2 +- 5 files changed, 12 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d6409f33f..dd079d394 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ Each release's notes. From 1.0.0 on, a major or minor release has hand-written n ## Unreleased +- Renovate holds ExpansionHunter at its pinned 5.0.0 image, the last Apache-2.0 release. Every bump, including a Bioconda rebuild, needs a licence review by hand. Upstream's default branch uses PolyForm Strict 1.0.0. + - The reports now get summary.json from the Rust `og` binary. Step 24, the Nextflow HTML_REPORT process and `generate-report.sh` collect it with `og summary collect` (through `bin/render_report.py`), and a run collects each sample once: `generate-report.sh` renders the text report from the summary.json step 24 wrote, and collects again when a file it reads is newer than that summary.json, or a file was added to or removed from a step folder since, or a file a section names as its source, the run status or a table in `bin/` or `assets/` is gone. When it renders from step 24's summary.json, the text report's generated time is step 24's, the same as the HTML report's; it was the minute of its own collect. When it collects again, the text report carries the new collect's values and time, and the HTML report keeps step 24's until step 24 runs again, as before. The deliberate fixes the `og summary collect` entries below list now reach the reports; apart from them, nothing in summary.json or either report changes. `bin/collect_summary.py` keeps only what the report renderer (`bin/render_report.py`), the couple report, the results index and the live view import from it. The Python modules that only the old collector imported are gone: `bin/pgx_outside_calls.py`, `ancestry_fine_report.py`, `clip_check.py`, `annotsv_postfilter.py`, `chip_report.py`, `traits_report.py`, `run_delta.py` and `genotype_rules_indels.py`. On the synthetic demo sample (4.7 million VCF records) the collect takes 1.1 s of CPU and 12-15 MB, where the Python took 7.8-8.0 s and 34-37 MB. What changes on purpose: - The walk of the sample folder leaves out `/nextflow`. `run-all.sh` launches Nextflow there, and its `work/` holds every task's files and staged links. No section reads a file there. With 200,000 files under it, the collect took 67 MB walked and 14 MB skipped. - The sections other than the sample VCF are read one at a time, while the sample VCF is read on its own thread, so the peak memory is near the biggest section's and not the sum of all of them. With every side table 50 times its demo size, the peak footprint fell from 119-130 MB to 105-112 MB (the Python: 100-102 MB), with the same CPU time. diff --git a/docs/updating.md b/docs/updating.md index 17f2561f9..44b0d595e 100644 --- a/docs/updating.md +++ b/docs/updating.md @@ -63,6 +63,7 @@ Each hold is a rule in `renovate.json` with its reason in the rule's `descriptio | Pin | Rule | Why | When it ends | |---|---|---|---| +| ExpansionHunter 5.0.0 | disabled, including Bioconda rebuilds | [5.0.0 is Apache-2.0](https://github.com/Illumina/ExpansionHunter/blob/v5.0.0/LICENSE.txt). [Upstream's default branch uses PolyForm Strict 1.0.0](https://github.com/Illumina/ExpansionHunter/blob/master/LICENSE.txt) | read the proposed tag's licence and the image's conda-meta licence before changing the hold. A PolyForm Strict version must become opt-in with a notice, like Cyrius | | pypgx 0.26.0 | only `0.26.0--` tags | 0.27.0 pulled pandas 3.0 and broke every gene while its smoke test still passed | upstream releases the pandas fix and a pypgx-bundle tag of the same version exists; move `PYPGX_BUNDLE_VERSION` with it | | plink2 2.00a5.10 | disabled | the only versioned tag `pgscatalog/plink2` ships, and the build pgsc_calc uses | a newer tag from that publisher | | Manta, Strelka2, duphold, Octopus, GRIDSS | disabled | legacy: archived or quiet upstream | bump by hand after a run on the fixture | @@ -74,6 +75,7 @@ Each hold is a rule in `renovate.json` with its reason in the rule's `descriptio ## What to check by hand +- **Image licences**: before any image bump, read the upstream licence at the proposed tag. Check the licence fields of every package in the image's conda-meta files too. A version or build number alone does not prove the licence. Keep ExpansionHunter held until that review is complete. - **PharmCAT**: diff the diplotype table on the HG002 fixture between the old and the new image, and run the CPIC parser tests on a `report.json` from the new version. - **VEP major**: move `VEP_CACHE_RELEASE` to the new major and `GENCODE_RELEASE` (`scripts/lib/common.sh`) to the new major minus 66 in the same PR, download the new cache (about 26 GB) and rerun step 13. CI cannot run the offline cache. - **PCGR/CPSR**: move `PCGR_DATA_BUNDLE` and `PCGR_VEP_CACHE_RELEASE` with the image, and rerun step 17 on a sample with an earlier result. CI does not run CPSR on a sample. The image test checks that `cpsr --help` still lists every flag step 17 passes, that `cpsr --version` names the release line of the image tag and the `pcgrr` R package its full release, and runs the bundled VEP with `--database` on 50 variants. diff --git a/docs/versions.md b/docs/versions.md index 539d5a4a7..baa298b27 100644 --- a/docs/versions.md +++ b/docs/versions.md @@ -42,7 +42,7 @@ This page is generated by `scripts/ci/gen-versions-doc.sh` from `versions.env` a | Specialized analysis | hipstr | `HIPSTR_IMAGE` | `quay.io/biocontainers/hipstr` | `0.7--hcf09f9e_1` | optional: step 45 (opt-in, Y-STR repeat counts of male samples) | | Specialized analysis | t1k | `T1K_IMAGE` | `quay.io/biocontainers/t1k` | `1.0.10--h5814d7d_0` | | | Specialized analysis | optitype | `OPTITYPE_IMAGE` | `quay.io/biocontainers/optitype` | `1.5.0--pyhdfd78af_1` | optional: step 08 with HLA_QC=true (opt-in second HLA caller, class I) | -| Specialized analysis | expansionhunter | `EXPANSIONHUNTER_IMAGE` | `quay.io/biocontainers/expansionhunter` | `5.0.0--hc26b3af_5` | | +| Specialized analysis | expansionhunter | `EXPANSIONHUNTER_IMAGE` | `quay.io/biocontainers/expansionhunter` | `5.0.0--hc26b3af_5` | hold: last Apache-2.0 release; upstream now PolyForm Strict; Renovate disabled, review the licence before any bump | | Specialized analysis | goleft | `GOLEFT_IMAGE` | `quay.io/biocontainers/goleft` | `0.2.6--he881be0_1` | | | Specialized analysis | mosdepth | `MOSDEPTH_IMAGE` | `quay.io/biocontainers/mosdepth` | `0.3.14--h87be163_2` | | | Specialized analysis | parascopy | `PARASCOPY_IMAGE` | `quay.io/biocontainers/parascopy` | `1.19.0--py311h98bf483_0` | optional: step 35 (opt-in, SMN1/SMN2 copy number) | diff --git a/renovate.json b/renovate.json index 14f063c4e..426ca025a 100644 --- a/renovate.json +++ b/renovate.json @@ -170,6 +170,12 @@ ], "packageRules": [ + { + "description": "Hold: ExpansionHunter 5.0.0 is the last Apache-2.0 release. Upstream's default branch uses PolyForm Strict 1.0.0. All updates, including Bioconda rebuilds, need a licence review by hand before this hold is changed (docs/updating.md).", + "matchDatasources": ["docker"], + "matchPackageNames": ["quay.io/biocontainers/expansionhunter"], + "enabled": false + }, { "description": [ "Biocontainer tags are --_. Docker versioning reads the part after '--' as a suffix that must stay the same, and the hash changes between versions, so most of these pins would never see a newer tag.", diff --git a/versions.env b/versions.env index 304024130..2900e700e 100644 --- a/versions.env +++ b/versions.env @@ -74,7 +74,7 @@ YLEAF_IMAGE="quay.io/biocontainers/yleaf:4.1.7--pyhdfd78af_0" # optional: step HIPSTR_IMAGE="quay.io/biocontainers/hipstr:0.7--hcf09f9e_1" # optional: step 45 (opt-in, Y-STR repeat counts of male samples) T1K_IMAGE="quay.io/biocontainers/t1k:1.0.10--h5814d7d_0" OPTITYPE_IMAGE="quay.io/biocontainers/optitype:1.5.0--pyhdfd78af_1" # optional: step 08 with HLA_QC=true (opt-in second HLA caller, class I) -EXPANSIONHUNTER_IMAGE="quay.io/biocontainers/expansionhunter:5.0.0--hc26b3af_5" +EXPANSIONHUNTER_IMAGE="quay.io/biocontainers/expansionhunter:5.0.0--hc26b3af_5" # hold: last Apache-2.0 release; upstream now PolyForm Strict; Renovate disabled, review the licence before any bump GOLEFT_IMAGE="quay.io/biocontainers/goleft:0.2.6--he881be0_1" MOSDEPTH_IMAGE="quay.io/biocontainers/mosdepth:0.3.14--h87be163_2" PARASCOPY_IMAGE="quay.io/biocontainers/parascopy:1.19.0--py311h98bf483_0" # optional: step 35 (opt-in, SMN1/SMN2 copy number) From 65d76e95ae9dc76287a0415faebb23fb6dbf63ed Mon Sep 17 00:00:00 2001 From: GeiserX <9169332+GeiserX@users.noreply.github.com> Date: Sun, 15 Nov 2026 22:05:54 +0100 Subject: [PATCH 2/2] docs(updating): scope the conda-meta licence check to Bioconda images --- docs/updating.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/updating.md b/docs/updating.md index 44b0d595e..73fb46ac7 100644 --- a/docs/updating.md +++ b/docs/updating.md @@ -75,7 +75,7 @@ Each hold is a rule in `renovate.json` with its reason in the rule's `descriptio ## What to check by hand -- **Image licences**: before any image bump, read the upstream licence at the proposed tag. Check the licence fields of every package in the image's conda-meta files too. A version or build number alone does not prove the licence. Keep ExpansionHunter held until that review is complete. +- **Image licences**: before any image bump, read the upstream licence at the proposed tag. For a Bioconda image, also check the licence fields of every package in its conda-meta files. A version or build number alone does not prove the licence. Keep ExpansionHunter held until that review is complete. - **PharmCAT**: diff the diplotype table on the HG002 fixture between the old and the new image, and run the CPIC parser tests on a `report.json` from the new version. - **VEP major**: move `VEP_CACHE_RELEASE` to the new major and `GENCODE_RELEASE` (`scripts/lib/common.sh`) to the new major minus 66 in the same PR, download the new cache (about 26 GB) and rerun step 13. CI cannot run the offline cache. - **PCGR/CPSR**: move `PCGR_DATA_BUNDLE` and `PCGR_VEP_CACHE_RELEASE` with the image, and rerun step 17 on a sample with an earlier result. CI does not run CPSR on a sample. The image test checks that `cpsr --help` still lists every flag step 17 passes, that `cpsr --version` names the release line of the image tag and the `pcgrr` R package its full release, and runs the bundled VEP with `--database` on 50 variants.