Skip to content

test(sv,clinvar,str): check the SV, ClinVar and repeat tools in cargo test - #451

Open
geiresx wants to merge 7 commits into
GeiserX:mainfrom
geiresx:pgp/port-tests-sv-clinvar-str
Open

geiresx wants to merge 7 commits into
GeiserX:mainfrom
geiresx:pgp/port-tests-sv-clinvar-str

Conversation

@geiresx

@geiresx geiresx commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

Part of #411. Settles owner decision D1 for this package: Python must go for 1.0.0.

Problem

Five Python files tested the SV, ClinVar and repeat tools by running the og binary: og sv postfilter, og sv inv-filter, og sv svlen-fix, og sv add-callers, og clinvar join-adjacent, og clinvar hits, og clinvar deep-intronic, og str catalog and the RFC1 goldens of og str motif. That is about 1,880 lines of Python that test Rust code. They ran outside cargo test, so a change to a crate did not run its own checks in the crate's test run.

Fix

The checks move into the crates that own the tools, as integration tests. They keep the same inputs, the same assertions and the same red controls:

Python check Now
tests/test_annotsv_postfilter.py (exon edges, population AF, caller, consensus support, the command line, rule table, depth check by sex and PAR, and their controls) rust/crates/og-steps/tests/sv_postfilter.rs
tests/test_annotsv_sv_inputs.py (inv-filter, svlen-fix, add-callers, Delly read depth, and their controls) rust/crates/og-steps/tests/sv_inputs.rs
tests/test_clinvar_join_adjacent.py checks 1 to 7 (the join scenarios, the window's edge, the tag pass) rust/crates/og-clinvar/tests/port_tests_sv_clinvar_str_join.rs
the same file, check 8 (the fixture delins matched once) og's part on tests/fixtures/vcf/combined_allele_{sample,clinvar}.vcf, read in place, in the same Rust file. The real step 06 with bcftools on these fixtures is the e2e case clinvar-combined-allele-1, which asserts the same hits.
the same file, check 9 (a split 1/2 poly-A call, the always-report list) og clinvar hits and og clinvar deep-intronic on the calls as step 06's bcftools writes them, in the same Rust file. The real 1/2 split through step 06 stays in tests/test_isec_columns.sh.
tests/test_str_catalog.py (pinned STRchive files and their checksum, the converter, Stranger on the catalog, the extra repeats against GRCh38, and their controls) rust/crates/og-setup/tests/strchive_catalog.rs
tests/test_rfc1_motif.py (what the RFC1 goldens must say) rust/crates/og-bam/tests/motif_meaning.rs

The Python files of the first four rows are deleted. Decisions, each the smallest change that keeps main true:

  • tests/test_rfc1_motif.py stays for now. Three e2e cases (expansions-and-rfc1-1-motif, str-catalog-1-strchive, nextflow-report-parity-1-sections) and rust/crates/og-bam/tests/make_parity.py import its read sets. test(e2e): build genotype and repeat e2e inputs without inline Python #442 moves two of those cases off it; the other importers belong to later lanes. The file goes when its last importer does.
  • tests/test_str_ranges.py imported three helpers from tests/test_str_catalog.py. It now holds them itself, unchanged, so the catalog file can go. A later lane moves test_str_ranges.py into Rust.
  • The STRchive tests download the pinned files from GitHub, read GRCh38 from NCBI and run Stranger in docker, as the Python did. They run where CI is set (Guard's rust job) or with OG_NET_TESTS=1. Elsewhere each prints that it skipped. In CI a failed download or a missing docker fails the test.
  • Running the real step 06 in docker stays with e2e and test_isec_columns.sh. Guard's cargo test runs before og is built for the containers, so a cargo test cannot run step 06.

No output, option or file name changes.

Files outside the package list, edited only to keep docs true to the deletion: tests/test_str_ranges.py (the helpers), docs/06-clinvar-screen.md, docs/09-str-expansions.md, docs/testing.md, tests/fixtures/vcf/README.md and tests/parity/gen/clinvar-cases.sh (each named a deleted file), and CHANGELOG.md (one line).

How it was checked

  • The 20 moved tests pass: cargo test -p og-steps --test sv_postfilter --test sv_inputs, -p og-clinvar --test port_tests_sv_clinvar_str_join, -p og-bam --test motif_meaning, and -p og-setup --test strchive_catalog with OG_NET_TESTS=1 (Stranger 0.10.2 in docker, the pinned STRchive files and NCBI GRCh38). cargo fmt --check, cargo clippy --all-targets --all-features -D warnings and cargo doc -D warnings are clean.
  • tests/test_str_ranges.py passes with a fresh og and OG_REQUIRED=1.
  • Red first: commit ac45187 planted one fault per moved test (the join window at 0, a half call worded as het, the Delly depth gate off, no INVDUP_CANDIDATE, a renamed ORIG_SVLEN header, CNVpytor calls named cnv, the MHC row gone, a far pathogenic minimum and short-read limit in og str catalog, UGT1A1's region shifted, three RFC1 goldens rewritten). Guard's cargo test --workspace --no-fail-fast went red in every moved file: sv_postfilter 5 of 5, sv_inputs 4 of 4, port_tests_sv_clinvar_str_join 4 of 4, motif_meaning 3 of 3, strchive_catalog 3 of 4 (the pinned-checksum test has no planted fault). https://github.com/GeiserX/OwnGenome/actions/runs/43745446386/job/132907615914 The next commit, 7863122, reverts it; its tree equals the tree before the plant.
  • Green on the final head 7863122: Guard's rust job ran all 20 moved tests and passed, with the STRchive tests on the network and Stranger in docker (strchive_catalog 4 of 4 in 6 s). https://github.com/GeiserX/OwnGenome/actions/runs/43749275443/job/132935403534 The other fast checks (unit tests, parity, fake-docker, Rust, Lint, Nextflow, Docs, Container Test, Personal data) are green on the same head.

Closes bead D1 for this package (the Python tests of the SV, ClinVar and repeat tools).

…rs in cargo test

The two Python files ran the og binary on made-up AnnotSV tables and SV
VCFs. Their checks move, by meaning and with the same inputs, into
og-steps' integration tests, and the Python files go.

Bead: D1
… in cargo test

The join scenarios, the tag pass, the fixture delins and the split poly-A
call move into og-clinvar's integration tests, which read
tests/fixtures/vcf/combined_allele_*.vcf in place. The real step 06 run on
the same fixtures stays with the e2e case clinvar-combined-allele-1, and
the real 1/2 split with tests/test_isec_columns.sh.

Bead: D1
…ldens in cargo test

test_str_catalog.py moves into og-setup's integration tests: the pinned
STRchive files and their checksum, the converter, Stranger on the built
catalog and the extra repeats against GRCh38. They download and run docker,
so they run in CI (CI set) or with OG_NET_TESTS=1. The meaning checks of
test_rfc1_motif.py on og str motif's goldens move into og-bam's tests.
test_str_ranges.py keeps the three helpers it imported from
test_str_catalog.py, so test_str_catalog.py can go.

Bead: D1
rustfmt and clippy on the moved tests, the CHANGELOG line, and the docs
and comments that named the deleted files.

Bead: D1
Each planted change must turn the moved cargo tests red in Guard's rust
job: the join window at 0, the 1/. wording as het, Delly's depth gate
off, no INVDUP_CANDIDATE, a renamed ORIG_SVLEN header, CNVpytor calls
named cnv, a 10% common-AF line, the MHC row gone, a far pathogenic
minimum and short-read limit in og str catalog, UGT1A1's region shifted,
and three RFC1 goldens rewritten.

Bead: D1
@coderabbitai

coderabbitai Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 25 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 9078a1d and 0a70ffb.

Files ignored due to path filters: 1

  • tests/parity/gen/clinvar-cases.sh is excluded by !**/gen/**
Files selected for processing: 15

  • CHANGELOG.md
  • docs/06-clinvar-screen.md
  • docs/09-str-expansions.md
  • docs/testing.md
  • rust/crates/og-bam/tests/motif_meaning.rs
  • rust/crates/og-clinvar/tests/port_tests_sv_clinvar_str_join.rs
  • rust/crates/og-setup/tests/strchive_catalog.rs
  • rust/crates/og-steps/tests/sv_inputs.rs
  • rust/crates/og-steps/tests/sv_postfilter.rs
  • tests/fixtures/vcf/README.md
  • tests/test_annotsv_postfilter.py
  • tests/test_annotsv_sv_inputs.py
  • tests/test_clinvar_join_adjacent.py
  • tests/test_str_catalog.py
  • tests/test_str_ranges.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Reverts ac45187. The planted faults turned the moved tests red in Guard's
rust job on that head; this head carries none of them.

Bead: D1
@geiresx

geiresx commented Nov 16, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants