Skip to content

Separate server construction from transport, and share argument validation - #28

Merged
adamjohnwright merged 3 commits into
mainfrom
feat/server-factory
Sep 14, 2026
Merged

adamjohnwright merged 3 commits into
mainfrom
feat/server-factory

Conversation

@adamjohnwright

Copy link
Copy Markdown
Contributor

Stacked on the #25#26 → deps chain. Retarget to main as those merge.

Harvests #5 by @adidev001.

#5 predates the current vitest suite, so its node:test harness and dist-test/ build are superseded. Two ideas in it are worth having, and they are taken here with credit.

A server factory

createServer() builds a fully-registered server with no transport attached; index.ts becomes the stdio entrypoint that calls it. Construction no longer happens at module scope, so importing the entrypoint no longer starts a server.

This is not only about testability. A hosted deployment has to serve Streamable HTTP from the same registrations, and it needs one server per session — neither is possible while the only instance is a module-scope constant. The open hosting question gets easier to answer with this in place.

Shared argument validation

A blank or whitespace-only argument is never a useful request: an empty q reaches the Content Service as either an error or a request for everything, and an empty analysis token produces a 404 that reads like the analysis expired.

nonEmptyString in src/schemas.ts trims and rejects, so the model gets a message it can act on instead of a confusing service error. 62 argument schemas across seven modules now share it, keeping the existing 2048-character cap.

Tests

Adapted from the registration tests in #5. These use the real registration path, not the fake server the formatter tests use, so a tool that fails to register is caught.

One asserts the graph tools stay absent while NEO4J_URI is unset — Principle IV of the constitution. Verified to actually fail when the gate is opened, rather than passing vacuously:

× keeps the graph tools behind the NEO4J_URI gate
  AssertionError: expected 'bolt://localhost:7687' to be falsy

Verification

  • npm run check: passes, 64 tests
  • npm run sweep: all 53 tools called, unchanged

🤖 Generated with Claude Code

adamjohnwright and others added 2 commits September 14, 2026 15:29
…ation

Harvested from #5 by @adidev001, which predates the current vitest suite. The
node:test harness and dist-test build in that PR are superseded, but two ideas
in it are worth having.

**A server factory.** `createServer()` builds a fully-registered server with no
transport attached, and `index.ts` becomes the stdio entrypoint that calls it.
Construction no longer happens at module scope, so importing the entrypoint
does not start a server.

This is not only about testability. A hosted deployment has to serve Streamable
HTTP from the same registrations, and it needs one server per session -- neither
is possible while the only instance is a module-scope constant. The open
hosting question gets easier to answer with this in place.

**Shared argument validation.** A blank or whitespace-only argument is never a
useful request: an empty `q` reaches the Content Service as either an error or
a request for everything, and an empty analysis token produces a 404 that reads
like the analysis expired. `nonEmptyString` in src/schemas.ts trims and
rejects, so the model gets a message it can act on instead of a confusing
service error. 62 argument schemas across seven modules now share it, keeping
the existing 2048-character cap.

Tests for the factory are adapted from the registration tests in #5. One of them
asserts the graph tools stay absent while NEO4J_URI is unset -- Principle IV,
and verified to fail when the gate is opened rather than passing vacuously.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The sweep added alongside the Spec Kit work greps rendered output for
"undefined", "[object Object]" and empty bodies. Testing it against a
deliberately reintroduced bug showed it catches the loud failures and misses
the quiet one -- the kind this repo actually shipped.

Reverting the search_facets fix and re-running produced:

    no suspicious output
    EXIT: 0

because a facets response that has silently dropped every facet still renders a
heading, a total, and "*No facets available.*". Three plausible lines, no
marker to grep for, and a confident report that there was nothing to report.
That is exactly the bug that went unnoticed for months, and the tool built to
find it could not.

So the sweep now also checks, for 16 tools whose arguments are known to return
data, that the answer still contains what it should. With that in place the
same experiment reports:

    reactome_search_facets  [missing "### Types:", "### Species:"]

Two further problems found while testing the sweep against itself:

  - It would have failed every scheduled run from the first week. Eight tools
    were flagged only because the sweep sent a pathway stable ID to a tool that
    wants a complex and got an honest 404. Per-tool arguments fix the eight,
    and a reply that is the service reporting an error is now listed separately
    and does not fail the run -- Reactome answering 500 is not something a
    release of this package can fix. A job that is always red teaches people to
    ignore it.

  - A typo in an expectation's tool name would have silently verified nothing.
    The run now prints how many expectations were actually checked and warns
    about any that name a tool which does not exist.

Also completes the nonEmptyString migration: 14 more argument schemas were
written across several lines by Prettier and so were missed by the first pass.
Only the definition itself now spells out the constraint.

Coverage thresholds raised 44 -> 50 to match the server-factory tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@adamjohnwright

Copy link
Copy Markdown
Contributor Author

Adversarial review of this stack found a flaw in the sweep itself

The live sweep added in #26 greps rendered output for undefined, [object Object] and empty bodies. I tested it by deliberately reintroducing the bugs it was built to find.

It catches the loud ones. It misses the quiet one.

Reverting the search_facets fix and re-running:

no suspicious output
EXIT: 0

A facets response that has silently dropped every facet still renders a heading, a total, and *No facets available.* — three plausible lines, no marker to grep for, and a confident report that there was nothing to report. That is precisely the bug this repo shipped for months, and the tool built to find it could not.

So the sweep now also asserts, for 16 tools whose arguments are known to return data, that the answer still contains what it should. Same experiment, after:

1 suspicious:
  reactome_search_facets  [missing "### Types:", "### Species:"]
EXIT: 1

And the loud case still works — reintroducing the search_suggest bug gives Cannot read properties of undefined (reading 'map'), exit 1.

Two more problems, found by testing the tool against itself

  • The scheduled workflow would have been red from week one. Eight tools were flagged only because the sweep sent a pathway stable ID to a tool that wants a complex and got an honest 404. Per-tool arguments fix those, and a reply that is the service reporting an error is now listed separately and does not fail the run — Reactome answering 500 is not something a release of this package can fix. A job that is always red teaches people to ignore it, which is the failure the constitution describes about CI being red for five months.

  • A typo in an expectation's tool name would have silently verified nothing — the same class of bug as everything else in this stack. The run now prints how many expectations were actually checked and warns about any naming a tool that does not exist.

Also in this commit

The nonEmptyString migration was incomplete: 14 more argument schemas had been split across lines by Prettier and so were missed by the first pass. Only the definition itself now spells out the constraint.

Coverage thresholds raised 44 → 50 to match the server-factory tests.

Current state: npm run check passes, 64 tests, 0 lint problems, 0 vulnerabilities, sweep green at exit 0 with all 53 tools called and 16 content expectations verified.

Note: this hardens the sweep introduced in #26 rather than amending it there, to avoid rebasing a four-deep stack. The end state on main is the same once the stack lands.

001 documents what was found and decided about response shapes -- including
that marker-grepping missed the quiet failure and what replaced it.

002 states the transport and hosting question rather than answering it: the
server factory is a prerequisite that landed, the rest is open, and the
boundaries that are already decided (no Neo4j from a deployed instance,
analysis in the Analysis Service) are written down so they are not relitigated.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Base automatically changed from deps/major-upgrades to main September 14, 2026 15:43
@adamjohnwright
adamjohnwright merged commit 36a5850 into main Sep 14, 2026
@adamjohnwright
adamjohnwright deleted the feat/server-factory branch September 14, 2026 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant