Skip to content

ADD --pam-only and --bgzip, for libraries short enough to match everywhere - #17

Merged
emattei merged 4 commits into
mainfrom
feat/pam-only
Nov 9, 2026
Merged

emattei merged 4 commits into
mainfrom
feat/pam-only

Conversation

@emattei

@emattei emattei commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator

A 9-mer matches hg38 about 24,000 times and carries an NGG at roughly 1,500
of them, so the annotation table is mostly rows nothing can be done with.
Two flags, both off by default.

--pam-only drops the matches whose genomic PAM does not match, as the
scan reads them, so neither the table nor memory holds them. On 500 9-mers
against chr21 that is 746,315 matches written as 20,398 rows.

Nothing else moves. ScanResult gains a per-row match count that
summarize takes, so n_matches keeps meaning every place the protospacer
occurs -- under this flag it is the only record that they exist. The
orientation verdict reads that count too: a guide that matched everywhere
without a PAM keeps none of its sites, and the empty list would otherwise
call it unmatched. A guide whose every site is dropped keeps the one empty
row a guide with no match gets.

--bgzip writes guide_mapping.bed.gz and its tabix index in place of
the plain BED, straight through BGZF so the plain text never reaches the
disk. The rows are already in coordinate order.

A bug found on the way. The Rust core's matches_pam rejects a genomic
N outright -- code has no value for one -- while base_matches accepts it
at an N of the pattern by design, and the pam_matches column has always
followed base_matches. n_no_pam came from the core, so on a reference
whose PAM is written NGG qc.tsv reported every site as having none while
the table marked them all TRUE. Both now ask pattern_matches, which makes
n_sites - n_no_pam the number of site rows a --pam-only run writes. The
reversed reading's orientation evidence still reads the core's flag; no
column reports it.

Tests: parity between a full run and a --pam-only one on the same library,
the genomic-N case, the counter against the column, a tabix region query,
and an empty index for a library with nowhere to cut.

🤖 Generated with Claude Code

emattei and others added 4 commits November 9, 2026 11:06
A spacer's matches grow as the genome over four to its length: a 9-mer
matches tens of thousands of places and can be cut at about one in
sixteen of them, so the table is mostly rows nothing can be done with.

The sites are dropped as the scan reads them, so neither the table nor
the memory holds them, and every match is still counted: ScanResult
gains a per-row count that summarize takes, so n_matches keeps meaning
every place the protospacer occurs. The orientation verdict reads that
count too -- a guide that matched everywhere without a PAM keeps none
of its sites, and the empty list would otherwise call it unmatched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Rust core's `matched` and the table's `pam_matches` part company over
a genomic N: the column reads one at an N of the pattern as a PAM, the
core does not. The flag asked the core, so on a reference that writes its
PAM as NGG every site was dropped though the table calls them all TRUE.

It now asks `pattern_matches`, the same question the column answers. The
counters still come from the core, so `n_no_pam` can disagree with the
column on such a reference; that is older than this flag.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The counter came from the Rust core's PAM verdict and pam_matches from
`base_matches`, and the two part company over a genomic N: the core's
`code` has no value for one, so `matches_pam` rejects it even at an N of
the pattern, while `base_matches` accepts it there by design. On a
reference that writes its PAM as NGG, qc.tsv reported every site as
having none while the table marked them all TRUE.

Both now ask `pattern_matches`, so n_sites - n_no_pam is the number of
site rows a --pam-only run writes. The reversed reading's evidence still
reads the core's flag: no column reports it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
guide_mapping.bed.gz and its tabix index in place of the plain BED, so a
browser or tabix can seek into a file too big to load. The rows are
already in coordinate order, and they go straight into the compressed
file through pysam's BGZFile, so the plain text never reaches the disk.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@emattei
emattei merged commit 2ec522d into main Nov 9, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant