diff --git a/CHANGELOG.md b/CHANGELOG.md index b3e8e4021..7f0f2c839 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ Each release's notes. From 1.0.0 on, a major or minor release has hand-written n ## Unreleased +- Step 12 now states and explicitly passes haplogrep3's existing `--hetLevel 0.9`: heterozygous calls with FORMAT/AF below 0.9 do not enter the haplogroup profile. The docs distinguish this from haplocheck's contamination check, the FASTA's AF 0.5 rule and the DeepVariant fallback's unread FORMAT/VAF. The Nextflow task stays in its work directory and uses the pinned image's bundled config and trees offline. The image smoke test checks an empty work directory and retained and excluded heteroplasmic markers. No output name, format or classification threshold changes. Part of #411 (lin-37, lin-40, D10). - ROH summaries now divide F_ROH by the GRCh38 length outside the bands of the assessed autosomes. Per-site rows identify those chromosomes, including chromosomes with no ROH. A new `roh.data.scope` records the chromosomes, denominator and scope source. Relationship labels and UPD flags require sites on all 22 autosomes and the full unmasked autosome length. Segment-only files use their observed chromosomes as an unverified lower bound and get neither label. Empty input has null F_ROH. Segment counts and the step 11 summary table keep their format. - A one-chromosome ROH flag now names regions and citations without syndrome names. Its note includes deletion, acquired loss of heterozygosity and shared ancestry as alternatives. The maternal and paternal JSON keys remain as `"-"` for reader compatibility. The attributed `assets/imprinted_regions.tsv` is embedded at build time, replacing the duplicated Rust table. The container builder mounts it, and the source stamp includes its blob hash, so a table edit makes an old binary stale. The stamp format and `tree-hash` command name stay the same. Every listed chromosome uses the same lookup, including chr11 and chr15. A region hint does not establish overlap with its imprinted locus or parental origin. Broad cytoband names remain reference labels, not diagnostic intervals. - The ROH collector now streams per-site rows instead of keeping the whole file in memory. On 1.1 million synthetic site rows, peak RSS fell from 65.7-67.0 MB to 3.0-3.5 MB. CPU time stayed near 0.2 seconds. Under shared host load, wall time ranged from 1.40-21.46 seconds before and 7.89-38.16 seconds after; these timings do not establish a speedup. diff --git a/docs/12-mito-haplogroup.md b/docs/12-mito-haplogroup.md index 5fc8b210f..6e1e7645e 100644 --- a/docs/12-mito-haplogroup.md +++ b/docs/12-mito-haplogroup.md @@ -23,8 +23,10 @@ Pinned in `versions.env`; [Image versions](versions.md) lists the current tags. The step reads, first match: -1. `${SAMPLE}/mito/${SAMPLE}_chrM_filtered.vcf.gz`, step 20's Mutect2 calls in mitochondrial mode. Mutect2 is the caller made for chrM: it reports each allele's fraction, so haplogrep3 sees heteroplasmies and haplocheck can run. The step keeps the PASS records (possible NuMTs are filtered by step 20) and splits multi-allelic records. -2. Otherwise the chrM records of `${SAMPLE}/vcf/${SAMPLE}.vcf.gz` (step 03). DeepVariant calls chrM as a diploid nuclear contig, so this is the fallback, and haplocheck does not run on it. +1. `${SAMPLE}/mito/${SAMPLE}_chrM_filtered.vcf.gz`, step 20's Mutect2 calls in mitochondrial mode. Mutect2 reports each allele's fraction. haplocheck reads these fractions to check for contamination. The step keeps the PASS records (possible NuMTs are filtered by step 20) and splits multi-allelic records. +2. Otherwise the chrM records of `${SAMPLE}/vcf/${SAMPLE}.vcf.gz` (step 03). DeepVariant calls chrM as a diploid nuclear contig, so this is the fallback, and haplocheck does not run on it. Its heterozygous calls carry FORMAT/VAF, which haplogrep3 does not read. Only its homozygous ALT calls contribute variants to the profile. + +haplogrep3 classifies with `--hetLevel 0.9` in both the script and Nextflow. For heterozygous calls with FORMAT/AF, it keeps ALT alleles at a fraction of 0.9 or more. Lower fractions do not enter its haplogroup profile. Homozygous ALT calls are kept regardless of AF. The threshold does not change haplocheck's contamination check or step 20's heteroplasmy report. Run step 20 before step 12 for the Mutect2 input. The Nextflow pipeline does the same: with `mito_variants` in `--tools` (it is in the default set), `mito_haplogroup` reads `MITO_VARIANTS`' calls and runs `HAPLOCHECK`. @@ -46,7 +48,7 @@ docker run --rm --network none -v "${GENOME_DIR}:/genome" "${BCFTOOLS_IMAGE}" ba "bcftools view -f PASS ${M}/${SAMPLE}_chrM_filtered.vcf.gz | bcftools norm -m-any -Oz -o ${M}/${SAMPLE}_chrM_for_haplogroup.vcf.gz" docker run --rm --network none -v "${GENOME_DIR}:/genome" "${HAPLOGREP3_IMAGE}" \ - haplogrep3 classify --tree phylotree-fu-rcrs@1.2 \ + haplogrep3 classify --tree phylotree-fu-rcrs@1.2 --hetLevel 0.9 \ --input ${M}/${SAMPLE}_chrM_for_haplogroup.vcf.gz \ --output ${M}/${SAMPLE}_haplogroup.txt --extend-report @@ -65,7 +67,7 @@ docker run --rm --network none -v "${GENOME_DIR}:/genome" "${HAPLOCHECK_IMAGE}" Both reports (step 24 and the text report) show the haplogroup and the contamination status, or "not checked" when the input was the step 03 VCF, and the links of the next section built from the haplogroup. -The FASTA is the reference chrM (the rCRS, in GRCh38) with the sample's majority alleles applied: the Mutect2 calls with an allele fraction of 0.5 or more, or the haploid and homozygous ALT calls of the step 03 VCF. A position without a call reads as the reference, so the FASTA is only as complete as the chrM calls. A heteroplasmy below 0.5 is not in it. +The FASTA is the reference chrM (the rCRS, in GRCh38) with the sample's majority alleles applied: the Mutect2 calls with an allele fraction of 0.5 or more, or the haploid and homozygous ALT calls of the step 03 VCF. A position without a call reads as the reference, so the FASTA is only as complete as the chrM calls. A heteroplasmy below 0.5 is not in it. Mutect2 alleles at AF 0.5 to below 0.9 can therefore appear in the FASTA while absent from the haplogroup profile. ## Explore your branch further @@ -88,5 +90,5 @@ Names differ between trees. haplogrep3 uses PhyloTree 17 names (`phylotree-fu-rc ## Interpretation - Common European haplogroups: H, U, J, T, K, V, W, X - Output includes quality score (0-1): >0.9 = high confidence -- Discordant variants may indicate heteroplasmy (mixture of mtDNA types) +- Discordant variants alone do not establish heteroplasmy. Read step 20's allele fractions and haplocheck's contamination report alongside the haplogroup. - Contamination `YES` with a level of a few percent: compare with step 33's VerifyBamID2 estimate, which reads the nuclear genome. The two can differ when the mtDNA copy number differs between the two people's cells. diff --git a/modules/local/mito_haplogroup/main.nf b/modules/local/mito_haplogroup/main.nf index e15d83ea0..05093b9b8 100644 --- a/modules/local/mito_haplogroup/main.nf +++ b/modules/local/mito_haplogroup/main.nf @@ -92,20 +92,16 @@ process MITO_HAPLOGROUP { script: """ - # haplogrep3 reads haplogrep3.yaml and its trees from the working - # directory. In the task directory it finds neither and fetches the - # config from GitHub, which fails without network (and was an unpinned - # download before). So it runs in the directory of its binary, where the - # image keeps them, as tests/smoke/haplogrep3.sh does. - WD=\$PWD - cd "\$(dirname "\$(readlink -f "\$(command -v haplogrep3)")")" + # The pinned Bioconda build uses the config and trees beside its jar + # when the task directory has no haplogrep3.yaml. Stay in the task + # directory, with no network, as the image smoke test does. haplogrep3 classify \\ --tree phylotree-fu-rcrs@1.2 \\ - --input "\$WD/${chrm_vcf}" \\ - --output "\$WD/${meta.id}_haplogroup.txt" \\ + --hetLevel 0.9 \\ + --input "${chrm_vcf}" \\ + --output "${meta.id}_haplogroup.txt" \\ --extend-report REPORTED=\$( { haplogrep3 --version 2>&1 || true; } | awk '!v && match(\$0, /[0-9]+\\.[0-9]+(\\.[0-9]+)*/) { v = substr(\$0, RSTART, RLENGTH) } END { print (v != "" ? v : "unknown") }') - cd "\$WD" cat <<-END_VERSIONS > versions.yml "${task.process}": diff --git a/scripts/12-mito-haplogroup.sh b/scripts/12-mito-haplogroup.sh index f0380e986..e533832f1 100755 --- a/scripts/12-mito-haplogroup.sh +++ b/scripts/12-mito-haplogroup.sh @@ -9,9 +9,10 @@ # (docs/12-mito-haplogroup.md lists the routes; nothing is uploaded) # # Mutect2 in mitochondrial mode is the caller made for chrM: it reports each -# allele's fraction, so haplogrep3 sees heteroplasmies and haplocheck can look -# for two haplogroups mixed in one sample. DeepVariant calls chrM as a diploid -# nuclear contig, so its calls are the fallback, without the contamination check. +# allele's fraction. haplogrep3 keeps heterozygous calls with FORMAT/AF >= 0.9; +# haplocheck uses the heteroplasmies to look for two haplogroups mixed in one +# sample. DeepVariant calls chrM as a diploid nuclear contig, so its calls are +# the fallback, without the contamination check. set -euo pipefail SAMPLE=${1:?Usage: $0 } @@ -58,6 +59,7 @@ run_in \ "${HAPLOGREP3_IMAGE}" \ haplogrep3 classify \ --tree phylotree-fu-rcrs@1.2 \ + --hetLevel 0.9 \ --input "$(cpath "$CHRM")" \ --output "/genome/${SAMPLE}/mito/${SAMPLE}_haplogroup.txt" \ --extend-report diff --git a/tests/smoke/haplogrep3.sh b/tests/smoke/haplogrep3.sh index 7a086488c..3e686426d 100644 --- a/tests/smoke/haplogrep3.sh +++ b/tests/smoke/haplogrep3.sh @@ -1,11 +1,51 @@ # shellcheck shell=sh -# HAPLOGREP3_IMAGE row. haplogrep3 reads haplogrep3.yaml and its trees from -# the working directory; step 12 runs it in the image's own working -# directory, so the row moves to the directory of the binary first. -set -e -BIN=$(readlink -f "$(command -v haplogrep3)") -cd "$(dirname "$BIN")" +# HAPLOGREP3_IMAGE row. Start in an empty writable task directory. The pinned +# image finds its bundled config and trees beside the jar, without a network +# or a change into the binary directory (which can be read-only). +set -eu +WORK=$(mktemp -d /out/haplogrep3.XXXXXX) +trap 'rm -rf "$WORK"' EXIT +cd "$WORK" echo "running in $(pwd)" -haplogrep3 classify --tree phylotree-fu-rcrs@1.2 --input /in/mito.vcf.gz \ - --output /out/haplogroup.txt --extend-report +haplogrep3 classify --tree phylotree-fu-rcrs@1.2 --hetLevel 0.9 \ + --input /in/mito.vcf.gz --output /out/haplogroup.txt --extend-report cat /out/haplogroup.txt + +# Mutect2-style GT:AF calls: below, on and above the chosen threshold. +# A homozygous ALT call stays in the profile even with a low AF. +cat > fractions.vcf <<'VCF' +##fileformat=VCFv4.2 +##contig= +##FORMAT= +##FORMAT= +#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT S1 +chrM 750 . A G 50 PASS . GT:AF 1/1:0.05 +chrM 3243 . A G 50 PASS . GT:AF 0/1:0.05 +chrM 5000 . C T 50 PASS . GT:AF 0/1:0.95 +chrM 9000 . T C 50 PASS . GT:AF 0/1:0.60 +chrM 10000 . A G 50 PASS . GT:AF 0/1:0.90 +VCF +haplogrep3 classify --tree phylotree-fu-rcrs@1.2 --hetLevel 0.9 \ + --input fractions.vcf --output fractions.txt --extend-report +awk -F '\t' ' + NR == 1 { + for (i = 1; i <= NF; i++) { + gsub(/"/, "", $i) + if ($i == "Input_Sample") col = i + } + next + } + NR == 2 && col { + profile = " " $col " " + gsub(/"/, "", profile) + ok = profile ~ / 750G / && profile ~ / 5000T / && profile ~ / 10000G / \ + && profile !~ / 3243G / && profile !~ / 9000C / + } + END { + if (!ok) { + print "FAIL: haplogrep3 profile does not obey hetLevel 0.9" > "/dev/stderr" + exit 1 + } + print "PASS: AF 0.90 and 0.95 retained; AF 0.05 and 0.60 excluded; homozygous ALT retained" + } +' fractions.txt