Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .github/workflows/guard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -371,6 +371,46 @@ jobs:
cp -f "$saved" "$src"
trap - EXIT

# The PGx tests moved from Python into cargo test: each planted fault must
# fail the moved test that guards that rule (one per new test target).
- name: PGx test controls (a planted fault fails its moved test in each crate)
run: |
files="crates/og-steps/src/pgx_rules.rs crates/og-pgx/src/sections.rs crates/og-filters/src/mito.rs crates/og-filters/src/refblock.rs crates/og-bam/tests/data/parity/want/check-multimapped_with_alt/files/check.tsv"
saved="${RUNNER_TEMP}/pgx-controls"
mkdir -p "$saved"
for f in $files; do cp -f "$f" "$saved/$(basename "$f")"; done
trap 'for f in $files; do cp -f "$saved/$(basename "$f")" "$f"; done' EXIT
# plant FILE EXPR: apply one fault and fail if it changed nothing.
plant() {
cp -f "$1" "${RUNNER_TEMP}/pgx-before"
perl -pi -e "$2" "$1"
cmp -s "$1" "${RUNNER_TEMP}/pgx-before" && { echo "ERROR: no fault was planted in $1: $2"; exit 1; }
return 0
}
plant crates/og-steps/src/pgx_rules.rs 's/\} else if !same_pair\(&names, calls\) \{/} else if false \&\& !same_pair(\&names, calls) {/'
plant crates/og-steps/src/pgx_rules.rs 's/if !same_diplotype\(&pg, &cy\.0\) \{/if same_diplotype(\&pg, \&cy.0) {/'
plant crates/og-pgx/src/sections.rs 's/cyp2d6_band\(all\[0\] \* factor\)/cyp2d6_band(all[0])/'
plant crates/og-pgx/src/sections.rs 's/STEP36_GENES\.contains\(&c\.gene\.as_str\(\)\) && !confirmed \{/STEP36_GENES.contains(\&c.gene.as_str()) \&\& false \&\& !confirmed {/'
plant crates/og-filters/src/mito.rs 's/^pub const MIN_AF: f64 = 0\.95;/pub const MIN_AF: f64 = 0.94;/'
plant crates/og-filters/src/refblock.rs 's/^pub const MIN_GQ: i128 = 20;/pub const MIN_GQ: i128 = 19;/'
plant crates/og-bam/tests/data/parity/want/check-multimapped_with_alt/files/check.tsv 's/^status\tunreliable$/status\tok/'
log="${RUNNER_TEMP}/pgx-controls.log"
if cargo test --locked --no-fail-fast -p og-steps -p og-pgx -p og-filters -p og-bam \
--test pgx_hla_second_caller --test port_tests_pgx_cpic --test port_tests_pgx_mito \
--test port_tests_pgx_refblock --test cyp2d6_depth --test pgx_outside_calls \
--test port_tests_pgx_outside > "$log" 2>&1; then
cat "$log"
echo "ERROR: the moved PGx tests passed with the planted faults"
exit 1
fi
cat "$log"
for test in case2_optitype_disagrees_at_hla_b a_moderate_inhibitor_halves_the_activity_score \
case2_the_af_bound the_gq_threshold each_depth_table_gives_its_verdict_and_the_shares \
case3_both_agree_and_the_depth_check_passed case10_a_stale_report; do
grep -E "^test ${test} .*FAILED" "$log" || { echo "ERROR: $test did not catch its fault"; exit 1; }
done
echo "Control: each planted PGx fault failed the moved test that guards it."

- name: cargo test
run: cargo test --workspace --locked --no-fail-fast

Expand Down
4 changes: 2 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,8 +155,8 @@ Change its line in `versions.env`, plus the coupled data variable its comment na
### PharmCAT 3.x (pinned: 3.4.0)
- **Two-step workflow**: Preprocessor (`pharmcat_vcf_preprocessor` with `-refFna`) then main jar (`pharmcat.jar`). NOTE: since 3.0 the preprocessor script lost its `.py` extension and the Python package was renamed `preprocessor` → `pcat`. The old `-refFasta` flag is long gone.
- Preprocessor outputs `.preprocessed.vcf.bgz` (NOT `.vcf`).
- **3.x JSON changes vs 2.15.x**: `wildtypeAllele` → `referenceAllele`; the HTML report is **no longer emitted unless `-reporterHtml` is passed explicitly**. The `genes` map may be flat (`{gene -> data}`) or nested (`{source -> {gene -> data}}`). `sourceDiplotypes` (or `recommendationDiplotypes`) carry `allele1`/`allele2` objects with a `.name`. Both CPIC consumers (`scripts/27-cpic-lookup.sh` and `modules/local/cpic_lookup`) run one parser, `og pgx cpic-report` (`rust/crates/og-pgx`): it **auto-detects both shapes**, gives each gene a status (`normal`, `non-normal`, `ambiguous` when the possible diplotypes have different phenotypes, `not called`, `unclassified` when a called gene's phenotype says neither normal nor changed function, listed without drug guidance and not counted) and **fails loud**: a recognized report yielding zero genes is reported as a parse failure, never "all genes were successfully called". Guarded by `tests/test_cpic_parser.py`, the `pgx-parse` parity cases and `cargo test -p og-pgx` on real 3.2.0 and 3.4.0 reports of the HG002 fixture, `tests/fixtures/pharmcat/report-<version>.json`.
- Pipeline pinned to **3.4.0** (3.4.0 still bundles vcf-parser 0.3.1, so the `##` header rewrite in step 7 and the module stays). Before bumping, revalidate steps 7 and 27 end-to-end against a known sample — JSON structure and preprocessor flags change between major versions — and capture the new version's `report.json` as a parser fixture beside the others (a row in the test's `REAL_REPORTS`).
- **3.x JSON changes vs 2.15.x**: `wildtypeAllele` → `referenceAllele`; the HTML report is **no longer emitted unless `-reporterHtml` is passed explicitly**. The `genes` map may be flat (`{gene -> data}`) or nested (`{source -> {gene -> data}}`). `sourceDiplotypes` (or `recommendationDiplotypes`) carry `allele1`/`allele2` objects with a `.name`. Both CPIC consumers (`scripts/27-cpic-lookup.sh` and `modules/local/cpic_lookup`) run one parser, `og pgx cpic-report` (`rust/crates/og-pgx`): it **auto-detects both shapes**, gives each gene a status (`normal`, `non-normal`, `ambiguous` when the possible diplotypes have different phenotypes, `not called`, `unclassified` when a called gene's phenotype says neither normal nor changed function, listed without drug guidance and not counted) and **fails loud**: a recognized report yielding zero genes is reported as a parse failure, never "all genes were successfully called". Guarded by `cargo test -p og-pgx` (`tests/port_tests_pgx_cpic.rs` runs the verb, `tests/fixtures.rs` the library) on real 3.2.0 and 3.4.0 reports of the HG002 fixture, `tests/fixtures/pharmcat/report-<version>.json`, and by the `pgx-parse` parity cases.
- Pipeline pinned to **3.4.0** (3.4.0 still bundles vcf-parser 0.3.1, so the `##` header rewrite in step 7 and the module stays). Before bumping, revalidate steps 7 and 27 end-to-end against a known sample — JSON structure and preprocessor flags change between major versions — and capture the new version's `report.json` as a parser fixture beside the others (a row in `REAL_REPORTS` of `rust/crates/og-pgx/tests/port_tests_pgx_cpic.rs`).

### pgsc_calc (PRS and ancestry, steps 25 and 26)
- **pgsc_calc runs as its own Nextflow pipeline** (`PGSC_CALC_VERSION` in versions.env; the `PGSC_*_IMAGE` lines move with it). `setup.sh` unpacks the release into `tools/pgsc_calc-<release>`; with `--pgsc_calc` pointing there the pipeline runs it offline (without it the PRS task fetches the release), and `run-all.sh` passes it when it exists.
Expand Down
2 changes: 1 addition & 1 deletion assets/pgx_contraindications.tsv
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
# biallelic: the disorder is recessive, so the WARNING needs both copies
# changed (a homozygous or hemizygous genotype) or two variants in the gene;
# one heterozygous variant makes a carrier and gives a NOTE instead.
# tests/test_cpic_parser.py checks the shape of every row.
# cargo test -p og-pgx (tests/port_tests_pgx_cpic.rs) checks the shape of every row.
Gene Drug Names CPIC_level Note Pair_source Label_source Report_if
POLG valproic acid valproic,valproate,divalproex A/B The valproate label contraindicates it in people with mitochondrial disorders caused by POLG mutations, and in children under two suspected of a POLG-related disorder: valproate can cause acute liver failure in them. https://api.cpicpgx.org/v1/pair_view?genesymbol=eq.POLG https://dailymed.nlm.nih.gov/dailymed/drugInfo.cfm?setid=08a65cf4-7749-4ceb-6895-8f4805e2b01f any
ASL valproic acid valproic,valproate,divalproex B The valproate label contraindicates it in people with a urea cycle disorder: valproate can cause hyperammonemic encephalopathy in them. https://api.cpicpgx.org/v1/pair_view?genesymbol=eq.ASL https://dailymed.nlm.nih.gov/dailymed/drugInfo.cfm?setid=08a65cf4-7749-4ceb-6895-8f4805e2b01f biallelic
Expand Down
2 changes: 1 addition & 1 deletion assets/pgx_inhibitors.tsv
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
# moderate one. '-' for every other gene: CPIC gives no rule that converts
# their phenotype, so an inhibitor of one gets a note and no converted
# phenotype.
# tests/test_cpic_parser.py checks the shape of every row.
# cargo test -p og-pgx (tests/port_tests_pgx_cpic.rs) checks the shape of every row.
Drug Gene Strength Activity_score_factor Class_source Rule_source
bupropion CYP2D6 strong 0 https://www.fda.gov/drugs/drug-interactions-labeling/healthcare-professionals-fdas-examples-drugs-interact-cyp-enzymes-and-transporter-systems https://cpicpgx.org/guidelines/guideline-for-codeine-and-cyp2d6/
fluoxetine CYP2D6 strong 0 https://www.fda.gov/drugs/drug-interactions-labeling/healthcare-professionals-fdas-examples-drugs-interact-cyp-enzymes-and-transporter-systems https://cpicpgx.org/guidelines/guideline-for-codeine-and-cyp2d6/
Expand Down
2 changes: 1 addition & 1 deletion docs/27-cpic-lookup.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ PharmCAT produces detailed JSON and HTML reports, but finding the actionable par

## Tool

- `og pgx cpic-report`, the one PharmCAT reader of the pipeline, in OwnGenome's Rust binary `og` (`rust/crates/og-pgx`, [Rust code](rust.md)). The Nextflow `CPIC_LOOKUP` module and the unit test `tests/test_cpic_parser.py` run the same command. `scripts/setup.sh --og` builds `og`.
- `og pgx cpic-report`, the one PharmCAT reader of the pipeline, in OwnGenome's Rust binary `og` (`rust/crates/og-pgx`, [Rust code](rust.md)). The Nextflow `CPIC_LOOKUP` module runs the same command, and `cargo test -p og-pgx` checks it beside its code. `scripts/setup.sh --og` builds `og`.

## Docker Image

Expand Down
2 changes: 1 addition & 1 deletion docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -257,7 +257,7 @@ The first port is the reference mask of `setup.sh --masked-reference`. On a 254

On the whole GRCh38 analysis set (3.1 GB) `og` writes the file `GRC_MASKED_REF_SHA256` pins in 29 s on the same machine, 1.2 s of it CPU: writing the copy is what takes the time.

The next four ports are the VCF filters of steps 04, 05 and 07: `og sv inv-filter`, `og sv svlen-fix`, `og pgx refblock-fill` and `og pgx mito-merge`. Their cases run each oracle in the image the step used (`inv-filter` on the Manta image's Python 2.7, and on Python 3 too). `refblock-fill` reads a stream on stdin, so its cases stage a file named `stdin`. e2e does not shadow these four: the step scripts call them from the image's own working directory, `refblock-fill` reads stdin, which the shadow cannot replay, and `mito-merge` writes `--out` outside the call's directory. Their unit tests (`tests/test_annotsv_sv_inputs.py`, `tests/test_pgx_refblock_fill.py`, `tests/test_pgx_mito_merge.py`) and the three step 05 tests run `og`. The unit-tests job has no `og`, so there they skip with a note. The Guard job `rust` runs them with `OG_REQUIRED=1` after its musl builds, and there a skip fails.
The next four ports are the VCF filters of steps 04, 05 and 07: `og sv inv-filter`, `og sv svlen-fix`, `og pgx refblock-fill` and `og pgx mito-merge`. Their cases run each oracle in the image the step used (`inv-filter` on the Manta image's Python 2.7, and on Python 3 too). `refblock-fill` reads a stream on stdin, so its cases stage a file named `stdin`. e2e does not shadow these four: the step scripts call them from the image's own working directory, `refblock-fill` reads stdin, which the shadow cannot replay, and `mito-merge` writes `--out` outside the call's directory. `tests/test_annotsv_sv_inputs.py` and the three step 05 tests run `og`. The unit-tests job has no `og`, so there they skip with a note. The Guard job `rust` runs them with `OG_REQUIRED=1` after its musl builds, and there a skip fails. `refblock-fill` and `mito-merge` are checked by `cargo test -p og-filters` (`tests/port_tests_pgx_refblock.rs`, `tests/port_tests_pgx_mito.rs`). These tests call the verbs' library code in a child copy of the test binary, so they need no built `og`, do not read `OG_REQUIRED` and never skip.

The checks CI runs on the pipeline itself are ports too: `og dev contract` (the output contract), `og dev schema` (a `summary.json` against `tests/schema/summary.schema.json`) and `og dev report-only` (the report-only error strategy). Their cargo tests in `rust/crates/og-dev/tests/repo.rs` run them on this checkout and on copies with one planted break each, in the Guard job `rust`. `og dev schema` and the run check of `og dev contract` have parity cases; `--static` and `og dev report-only` read a whole checkout, which a case cannot stage, so their parity was shown on the checkout and the planted copies when they were ported. Each fixes a check that passed on what it should refuse: a schema default of another JSON type (`false` for `0`), `additionalProperties: false`, `true` against `enum: [1]`, NaN against `minimum`, a `$ref` to nothing (now exit 2, a broken schema), a `//` inside a string and a nested brace in a strategy closure. `tests/schema/validate.py` stays only for three Python test files that validate the summaries they build in-process, in the unit-tests job, which has no `og`; it follows the same rules. The fake-docker case `reports-stale-step` checks its `summary.json` with `og dev schema` in the job `rust`; the job `fake-docker` has no `og` and says it skipped that check.

Expand Down
2 changes: 1 addition & 1 deletion modules/local/cpic_lookup/main.nf
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
CPIC Lookup — Drug-gene recommendations from PharmCAT results
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Reads PharmCAT's report.json with og pgx cpic-report (bin/og, on the task
PATH; the same code scripts/27-cpic-lookup.sh and tests/test_cpic_parser.py
PATH; the same code scripts/27-cpic-lookup.sh and cargo test -p og-pgx
run) and writes the gene phenotypes and the medications PharmCAT's own report
matches to them. With the PYPGX summary it also writes the PharmCAT/pypgx
comparison (published to pypgx/, as the bash steps do) and warns, in the
Expand Down
Loading
Loading