Skip to content

fix(chip): state the CHIP detection limit from real per-base depth, not the average - #449

Open
geiresx wants to merge 6 commits into
GeiserX:mainfrom
geiresx:pgp/i411-chip-read-support-limit
Open

geiresx wants to merge 6 commits into
GeiserX:mainfrom
geiresx:pgp/i411-chip-read-support-limit

Conversation

@geiresx

@geiresx geiresx commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

Part of #411: claim trait-59, with owner decision D10 (a coding base is "callable" at 10 reads or more).

The problem

Step 49 gave each CHIP gene one detection limit, worked out from the gene's mean depth. A gene with a fifth of its coding bases unread then read covered, with almost the same limit as a gene read evenly. On main, DNMT3A with 8 exons at 33x and 2 exons at 0x gives 26.4 0.157 covered, and 27x everywhere gives 27.0 0.152 covered. The step line and both reports then said a clone above that VAF "is found 8 times in 10", from the limit at the median depth. A dark exon stayed hidden, and the wording overstated a read count as the caller's sensitivity.

The fix

  • og chip report keeps each gene's per-base depths (bases with no pileup line at 0). The limit is the smallest VAF at which a clone shows 3 reads with the change 8 times in 10, averaged over the gene's coding bases, each at its own depth. With one depth everywhere it equals the old limit.
  • A gene where no VAF reaches that power has no limit: partly_covered when some bases have 3 reads, too_shallow when none do. A gene with a fifth of its bases or more under 3 reads cannot reach 8 in 10.
  • The stated limit of a run is the median of the covered genes' own limits. The step line, og summary collect's median_limit and so both reports use it. Before, it was the limit at the median depth.
  • chip_depth.tsv gets two columns at the end: callable_fraction (share of coding bases with 10 reads or more, D10) and assessable_fraction (share of coding bases where a clone at the stated limit shows its reads). The first seven columns keep their names and order. summary.json's chip genes carry the two columns when the table has them.
  • The step line now reads "Read-support limit ... shows at least 3 reads with the change 8 times in 10, counted base by base. Mutect2 and its filters call fewer than that." It names each gene with no limit, with its share of bases read deep enough.

Output changes, as 1.0.0 allows: two columns added to chip_depth.tsv, a new partly_covered status, new detection_limit_vaf and median_limit values, and new wording in the step line. Nothing is renamed or removed.

Decisions taken in this lane:

  • The stated limit is the median of the per-gene limits, not a limit pooled over all bases. og summary collect only sees chip_depth.tsv, so this keeps the step line, summary.json and the reports on one figure.
  • assessable_fraction uses the run's stated limit, so it is empty when no gene is covered.
  • The report and live-page wording stay with the display lanes (i411-display-traits, i411-display-live). They read the same median_limit, now the honest figure.

Files outside the package list, each needed for CI to stay true: tests/parity/golden/chip/report-{calls,table,overlap,empty}/ (deliberate-fix goldens), tests/test_chip_report.py (3 checks pinned the old mean-depth behaviour), and modules/local/chip/main.nf (the stub header gets the two columns).

How it was checked

  • Red first: rust/crates/og-chip/tests/i411_chip_read_support_limit.rs on the test-only commit c28e011 fails all 4 tests in CI: https://github.com/GeiserX/OwnGenome/actions/runs/43744441741/job/132904521191 (main writes covered where partly_covered is due, 0.137 where 0.159 is due, and no share columns). The same tests pass on the fix.
  • New unit tests in og_chip::binom: one depth everywhere equals the old limit to 1e-12, an unread fifth leaves no limit (the power at a VAF near 1 rounds to exactly 1, so the share of deep bases is tested on its own), and the readable-at-a-VAF rule.
  • Parity, ORACLE_RUNNER=local scripts/ci/oracle-diff.sh: all 18 chip cases and all 15 summary-collect group c cases agree, with goldens for the deliberate fixes. The whole-file cases all and all-history get goldens for the new chip genes keys. The c generator copies the full folder with cp -Rp and gives each table the float cases rewrite the fixture's time, so those goldens do not depend on the clock (a first CI run caught two dates of the clock: https://github.com/GeiserX/OwnGenome/actions/runs/43747267494).
  • tests/test_chip_report.py, test_live_results.py, test_live_widgets.py and test_render_report.py pass with the new og.
  • chip-clonal-1 e2e: GATK's image is amd64-only and cannot run on this box, so I ran the depth half on public data. I took GIAB HG002 60x reads over the MANE coding exons of the six CHIP genes on the fixture contigs (pileup at MQ 20, BQ 10, duplicates out) and ran og chip report. DNMT3A: mean 52.7x, limit 0.081 (covered, under the case's 10%), callable_fraction 1.000, assessable_fraction 0.293. The case's new checks pass on that output and fail on main's output (no share columns, old "Detection limit" line). The full case runs in the e2e train.
  • Speed and memory, /usr/bin/time -l: on the HG002 input (21,277 depth lines) both old and new take 0.01 to 0.06 s and 3.2 MB. On 2 Mb of synthetic depth up to 5,000x, both take 0.50 s; peak memory goes from 5.0 MB to 5.3 MB.
  • cargo fmt --check, cargo clippy --workspace --all-targets -D warnings and cargo doc -D warnings are clean.

Beads: trait-59, D10.

A gene with a fifth of its coding bases unread reads covered with a limit
near an evenly read gene's, and a 30x run with uneven depth states the
limit at the mean. These tests fail on main.

Bead: trait-59
@coderabbitai

coderabbitai Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 14 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 9078a1d and bab3a34.

Files ignored due to path filters: 5

  • tests/parity/gen/summary-collect-c.sh is excluded by !**/gen/**
  • tests/parity/golden/chip/report-calls/S_chip_depth.tsv is excluded by !**/*.tsv
  • tests/parity/golden/chip/report-empty/S_chip_depth.tsv is excluded by !**/*.tsv
  • tests/parity/golden/chip/report-overlap/S_chip_depth.tsv is excluded by !**/*.tsv
  • tests/parity/golden/chip/report-table/S_chip_depth.tsv is excluded by !**/*.tsv
Files selected for processing: 18

  • CHANGELOG.md
  • docs/49-chip.md
  • modules/local/chip/main.nf
  • rust/crates/og-chip/src/binom.rs
  • rust/crates/og-chip/src/lib.rs
  • rust/crates/og-chip/src/report.rs
  • rust/crates/og-chip/tests/i411_chip_read_support_limit.rs
  • rust/crates/og-summary/src/collect/group_c/tables.rs
  • tests/e2e/chip-clonal-1.sh
  • tests/parity/golden/chip/report-calls/stdout
  • tests/parity/golden/chip/report-overlap/stdout
  • tests/parity/golden/chip/report-table/stdout
  • tests/parity/golden/summary-collect/all-history/all-history.json
  • tests/parity/golden/summary-collect/all/all.json
  • tests/parity/golden/summary-collect/c-float--inf/float--inf.json
  • tests/parity/golden/summary-collect/c-float-inf/float-inf.json
  • tests/parity/golden/summary-collect/c-full/full.json
  • tests/test_chip_report.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

og chip report now keeps each gene's per-base depths (unread bases at 0)
and finds the VAF at which the mean per-base power reaches 0.8. A gene
where no VAF reaches it reads partly_covered (some bases have 3 reads) or
too_shallow (none), with no limit. Two columns are added at the end of
chip_depth.tsv: callable_fraction (bases at 10 reads or more, D10) and
assessable_fraction (bases where a clone at the run's stated limit shows
its reads). The stated limit is the median of the covered genes' own
limits, in the summary line and in og summary collect's median_limit.

Bead: trait-59
Bead: D10
og chip report's depth table and summary line, and og summary collect's
median_limit, differ from the Python on purpose now: goldens for the
chip report cases and the summary cases that read a chip depth table.
The group c generator gives the full folder the two share columns and a
partly_covered gene, and copies it with its times so the float cases do
not depend on the clock. The Python chip test, the Nextflow stub header
and the chip-clonal-1 e2e case follow the new columns and wording.

Bead: trait-59
Bead: D10
The float cases rewrite six tables with awk and mv, which left them at
the generation time, so the goldens of c-float-inf and c-float--inf held
a date of the clock. Each rewritten table now gets the fixture's time.

Bead: trait-59
…pport-limit

# Conflicts:
#	CHANGELOG.md
#	tests/parity/golden/summary-collect/all-history/all-history.json
#	tests/parity/golden/summary-collect/all/all.json
#	tests/parity/golden/summary-collect/c-full/full.json

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants