Spec 005: live Reactome data and analysis, plus a Release 98 pre-flight - #208
Merged
Merged
Conversation
The three MCP PRs from @GovindhKishore, and the fact that most of what they are justified by is a cheaper problem. Their motivation is that MCP tools query live APIs "regardless of when embeddings were built". True, and it conflates two problems. The bundle is two releases behind -- installed Release95, reactome.org reports 97 -- and it was already two behind when it was built eight days ago, so this is not drift from age. Nothing rebuilt it against a current release. That needs bin/embeddings_manager and a schedule, not an MCP server, a subprocess and a second router. What MCP alone can do is analysis: enrichment, traversal, entity lookup. None of that is similarity search over stored text, and no amount of rebuilding produces it. That is the honest reason to take the work, and it is the smaller-sounding half. Costs, all verified rather than assumed: reactome-mcp describes itself as "just a prototype for now" and was last pushed 2026-07-01; #127 spawns the server as a subprocess with stdio pipes, in a container that now runs as non-root and spawns nothing today; and #142 adds a second LLM classifier beside the intent classifier that already routes, in a pipeline where cutting LLM calls from 21 to one was the point of spec 001. Recommendation is to rebuild the bundle first and independently, which removes staleness from MCP's justification and leaves the real case, then adopt #127 and #137 behind a flag with routing folded into the existing classifier rather than #142's sibling. Recorded in the checklist: nobody has run reactome-mcp from this repository, so its tools, latency and failure behaviour are taken from a PR description rather than observed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
D1 is answered: Release 98 is nearly done and new embeddings go to production then. So MCP is to be judged on analysis alone, which was the point of separating the two problems. Checked the rebuild path against this machine rather than assuming it works, and three things are not ready. Each fails partway through a long job rather than at the start. Disk: the bundle is 3.4G and there is 5.2G free on a 95%-full volume, so a second one does not fit comfortably -- and the old one cannot be deleted first because the running chatbot is serving from it. S3: there is no ~/.aws, no AWS_* in the environment and nothing in .env, so ls-remote, pull and push all fail with AccessDenied. CI assumes an AWS role by OIDC but only for the ECR image push, not the embeddings bucket. push is how a bundle reaches production, so this blocks the move, not just the build. Neo4j: make reads the Reactome graph and defaults to localhost, and nothing documents it. The connection has to be passed explicitly. What is ready is the part most likely to have rotted: every data_generation module imports cleanly on LangChain 1.x. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
adamjohnwright
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two things: the MCP decision, and a readiness check for the Release 98 rebuild you are about to do.
The MCP PRs are justified mostly by a cheaper problem
Their pitch is that MCP tools query live APIs "regardless of when embeddings were built". True — and two separate problems:
Staleness. Installed bundle is Release95;
reactome.orgreports 97. It was already two behind on the day it was built (2026-09-02), so this is not drift — nothing rebuilt it. That does not need MCP, it needs a rebuild. (D1 now answered: Release 98 is nearly done and new embeddings go to prod then.)Analysis. Enrichment, traversal, entity lookup — not similarity search over stored text, and no rebuild produces it. This is the part only MCP can do, and with staleness removed it is the whole case.
Costs, verified not assumed
reactome/reactome-mcpis Reactome's own, but self-describes as "just a prototype for now", last pushed 2026-07-01Recommendation: adopt #127 and #137 behind a flag, routing folded into the existing classifier rather than #142's sibling.
The Release 98 pre-flight — three things are not ready
I checked the rebuild path against this machine. Each of these fails partway through a long job rather than at the start.
~/.aws, noAWS_*, nothing in.env.ls-remotefails AccessDenied, sopushdoes too. CI assumes an AWS role by OIDC but only for the ECR image push, not the embeddings bucket.pushis how a bundle reaches production — this blocks the move, not just the build.makereads the Reactome graph, defaults tolocalhost:7687, documented nowhere. Must be passed explicitly.What is ready is the part most likely to have rotted: every
data_generationmodule imports cleanly on LangChain 1.x.deploy/rebuilding-embeddings.mdhas the sequence, and a reminder that a new bundle changes what reaches the model — sobin/retrieval_baselinebefore and after, per Article II.🤖 Generated with Claude Code