Ship the cofactor list with the pathway artifacts, derived not typed - #78
Merged
Merged
Conversation
A consumer deciding whether a perturbation may travel through ATP needs to know which nodes are ATP, and that knowledge has to travel with the networks. It did not, so both this repo and DeltaSignal kept their own copy, and the copies drifted. Audited against Release97, 6 of the 13 hand-written ids here are wrong: R-ALL-29438 commented "PPi" is GTP, R-ALL-29390 commented "Pi variant" is PXLP (pyridoxal 5'-phosphate), R-ALL-29360 commented "ADP variant" is NAD+, and R-ALL-217093 / R-ALL-110114 / R-ALL-29986 do not exist in the release at all. Nothing caught any of it, because the list is only ever read here. So derive it. `get_cofactor_species` returns every SimpleEntity whose ReferenceMolecule carries one of 28 curated ChEBI identifiers — 253 species at Release97, every compartment variant included. ChEBI identity is the only stable handle: a name query misses compartment variants and matches by substring, and stable ids go stale between releases. This list managed both. `export_cofactors` writes cofactors.csv into each pathway directory alongside the networks, carrying stable_id, molecule, chebi_id, name, in_network and reactome_release. Every known cofactor is listed rather than only the present ones, so a consumer can tell an empty intersection from a bundle that predates the file, and the file is written even when a pathway contains none. The networks themselves are unchanged: this adds an artifact, and the existing _COFACTOR_STIDS bridge suppression is left exactly as it is. Correcting those six entries changes which bridges are suppressed, which changes the networks, which needs its own A/B — a separate change, deliberately not bundled here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Found by adversarial review. `reactome_id_to_uuid` is built as `reactome_id_to_uuid[entity_uuid] = entity_stId` — it is uuid -> stId despite its name, which is why `_uuid_to_stable_id_map` exists and why every other consumer calls `.get(uuid)`. `export_cofactors` was the only new export not using that helper: it unpacked `for stable_id, uuid in ...items()`, binding the names to the wrong halves, then tested a stId against a set of uuids. That is never true, so `present` stayed empty and EVERY row of EVERY cofactors.csv would have shipped with in_network=0 — a file asserting that no pathway contains any cofactor. The same scan was wrong a second way: one stable id routinely maps to several uuids (the positional-decomposition silo — GPVI carries four separate GTP nodes), and a dict keyed by stable id holds only one of them, so a split entity was undercounted even in the right direction. Both existing tests passed the mapping in the direction production never uses, so they locked the bug in rather than catching it. The two new tests exercise both directions and a split entity, and both fail against the previous code — verified by reverting the fix. mypy and ruff clean. pytest: 955 passed; the 8 failures are identical before and after this change (6 are a local scipy stub for this box's Python 3.14, 1 needs Neo4j, 1 pre-exists on main). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A consumer deciding whether a perturbation may travel through ATP needs to know which nodes are ATP — and that knowledge has to travel with the networks. It didn't, so this repo and DeltaSignal each kept a copy, and the copies drifted.
Audited against Release97, 6 of the 13 hand-written ids in
_COFACTOR_STIDSare wrong:R-ALL-29438R-ALL-29390R-ALL-29360R-ALL-217093R-ALL-110114R-ALL-29986Nothing caught any of it, because the list is only ever read here.
What this does
Derives the list.
get_cofactor_species()returns everySimpleEntitywhoseReferenceMoleculecarries one of 28 curated ChEBI identifiers — 253 species at Release97, every compartment variant included. ChEBI identity is the only stable handle: a name query misses compartment variants and matches by substring ("phosphate"pulls in PXLP), and stable ids go stale between releases. This list managed both failure modes.Ships it with the artifacts.
export_cofactors()writescofactors.csvinto each pathway directory:Pull a pathway out of S3 and it carries its own answer, pinned to the release it came from. Every known cofactor is listed rather than only the present ones, so a consumer can distinguish an empty intersection from a bundle predating the file — and the file is written even when a pathway contains none.
What this deliberately does not do
The networks are unchanged. This adds an artifact. The existing
_COFACTOR_STIDSbridge suppression is left exactly as it is, defects and all: correcting those six entries changes which bridges are suppressed, which changes the networks, which needs its own A/B. That is a separate PR.Testing
tests/test_provenance_exports.pycovering the all-listed/flag-present contract and the write-even-when-empty casemypy src/clean,ruff check src/ bin/cleanpytest tests/— 977 passed. One pre-existing failure (test_main_edges_proportional_to_best_matches[Class_I_MHC…]) reproduces onmainunchanged and is unrelated.🤖 Generated with Claude Code