Cap tool response sizes, and fix what measuring them uncovered - #32
Merged
Merged
Conversation
An MCP tool result is spent from the caller's context window, and nothing here
bounded one. Measured against the live services, the sizes are driven by the ID
asked about rather than by anything the tool decides:
reactome_events_hierarchy 87,996 chars ~22,000 tokens
reactome_query (Metabolism) 61,336 chars ~15,300 tokens
reactome_pathway_contained_events 23,413 chars ~5,900 tokens
A single call could crowd out the conversation it was meant to inform. The
Cypher tool has had a total-size cap since it was written; the REST tools had
nothing.
**A cap, applied once.** `MAX_TOOL_RESPONSE_CHARS` (40,000, matching the Cypher
default, overridable by env) is enforced in the wrapper every handler already
passes through -- a per-tool guard is one somebody forgets to add to the
fifty-seventh tool. The cut is announced in the text: a model that cannot see it
was truncated will report the partial answer as the whole one.
**The root cause in events_hierarchy.** It capped at three top-level pathways
and then recursed every descendant of each. It now takes `max_depth` (default 3)
and `top_level_limit`, and says where it stopped and which tool goes deeper.
86 KB -> 16 KB.
**reactome_query.** Pretty-printed JSON cost ~23% in whitespace no model needs.
And truncating a JSON dump hands back invalid JSON while still spending the
whole budget, so an object too large to return is now described instead --
field names, shapes and sizes, with a pointer to the `attribute` argument.
Metabolism: 43,457 characters of severed object -> 1,327 characters of usable
map.
Three further bugs fell out of this, each found by checking the fix rather than
by assuming it:
- **`reactome_query`'s `attribute` argument had never worked.**
`/data/query/{id}/{attribute}` answers `text/plain`, the client asks for
`application/json`, and the service returns HTTP 406. Found because the
summary above advises using `attribute` -- advice that would have been
wrong. It now uses the text path that `contentClient.getText` already had.
- **The test harness never ran zod.** `fake.invoke()` passed raw arguments
straight to handlers, so defaults never materialised and invalid input was
never rejected. A test could pass while the real server requested
`/data/eventsHierarchy/undefined`, and no test had ever exercised the
`nonEmptyString` validation added last week. The harness now parses through
the registered schema, and four tests cover validation that was previously
untestable.
- **The sweep's "empty body" check counted lines.** Compact JSON is one very
long line, so a complete answer was reported as empty. It now checks length
too.
Lint caught two `String(unknown)` calls in the new summariser -- "[object
Object]" waiting to happen, the same quiet wrongness as the field-path bugs.
41 -> 81 tests. Coverage thresholds raised to match. Live sweep: 53 tools
called, 16 content expectations checked, no suspicious output.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An MCP tool result is spent from the caller's context window, and nothing here bounded one. Measured against the live services — the sizes are driven by the ID asked about, not by anything the tool decides:
reactome_events_hierarchyreactome_query(Metabolism)reactome_pathway_contained_eventsA single call could crowd out the conversation it was meant to inform. The Cypher tool has had a total-size cap since it was written; the REST tools had none.
A cap, applied once
MAX_TOOL_RESPONSE_CHARS(40,000 — matching the Cypher default, overridable by env) is enforced in the wrapper every handler already passes through for request context. A per-tool guard is one somebody forgets to add to the fifty-seventh tool.The cut is announced in the text. A model that cannot see it was truncated will report the partial answer as the whole one — the same class of quiet wrongness as a formatter reading the wrong field.
Root causes, not just the cap
events_hierarchycapped at three top-level pathways and then recursed every descendant of each. Now takesmax_depth(default 3) andtop_level_limit, and says where it stopped and which tool goes deeper. 86 KB → 16 KB.reactome_querypretty-printed its JSON, costing ~23% in whitespace no model needs. And truncating a JSON dump returns invalid JSON while still spending the whole budget — so an object too large to return is now described: field names, shapes, sizes, and a pointer to theattributeargument.43,457 characters of severed object → 1,327 characters of usable map.
Three bugs found by checking the fix rather than assuming it
reactome_query'sattributeargument had never worked./data/query/{id}/{attribute}answerstext/plain; the client asks forapplication/json; the service returns HTTP 406. Found only because the summary above advises usingattribute— advice that would have been wrong. It now uses the text pathcontentClient.getTextalready provided.The test harness never ran zod.
fake.invoke()passed raw arguments straight to handlers, so defaults never materialised and invalid input was never rejected. A test could pass while the real server requested/data/eventsHierarchy/undefined— and no test had ever exercised thenonEmptyStringvalidation added last week. The harness now parses through the registered schema; four tests cover validation that was previously untestable.The sweep's "empty body" check counted lines. Compact JSON is one very long line, so a complete answer was flagged as empty. It now checks length too.
Lint caught two
String(unknown)calls in my own new summariser —"[object Object]"waiting to happen.Verification
npm run checkgreen, coverage thresholds raised to match (53% statements)events_hierarchy86 KB → 16 KB,reactome_query60 KB → 1.3 KB on large objects🤖 Generated with Claude Code