S0089-cli-agent-3: publish an output schema per MCP tool and conform every result - #49
Merged
Conversation
…every result
Every advertised tool now declares an `outputSchema` (and a `title`), every
dispatch path returns structured content conforming to the schema its own tool
declared, and the text content block is the serialized JSON of that same
structured value.
The structured result carries `ok`, a `status` of success / verdict / failed,
the `exit` code, a stable `code` on a failed call, and the tool's own payload
under `data` (the parsed model and warnings, or a record stream for a
multi-record input; the validation verdict and findings; the value-free
structural summary; the converted Bundle). `status` is the declared property
that separates a negative verdict about the message from a call that produced
nothing, which the old `{ exit, ok }` pair could not express.
Both handlers in `src/mcp/server.ts` are explicit allow-lists, so both were
widened: `tools/list` now copies the title and the output schema, and
`tools/call` passes the whole structured value through instead of rebuilding
`{ exit, ok }`.
PHI posture: on a failed call every property of the structured result is drawn
from a fixed set (the outcome vocabulary, the exit-code contract, the
CLI_CODES registry), so no part of the caller's input can appear in it. The
diagnostic code is matched against the registry rather than lifted out of
stderr, and an unknown tool name is no longer echoed back, because a tool name
is caller-supplied text like any other argument.
Test coverage validates each emitted structured value against its own tool's
declared schema using a dependency-free checker (test/helpers) that refuses any
schema keyword it does not implement, with a negative control proving it can
fail; the text-block round trip; the advertised-schema assertions over the SDK
client (which validates the replies itself once tools/list has been called);
and the unhappy paths. The PHI-leak matrix now covers the agent surface.
No dependency was added or moved. Tool names, tool count, tools/list ordering,
input schemas, the exit-code contract and the verdict-vs-isError rule are
unchanged.
…his branch) `cli` main advanced from 1f26d41 to 9a17fed while this branch was open, adding the de-identification work. Three files conflicted. CHANGELOG.md and test/phi-leak.test.ts: both sides kept, they are additive. test/mcp-tools.test.ts: main's two new cases are kept, and one assertion in "`redact` is an unknown tool name, answered value-free" is updated rather than kept verbatim. It asserted the text block contains `unknown tool '<name>'`, and this branch deliberately stops echoing a caller-supplied tool name: the text block is now the serialization of the structured result, and the structured result carries no part of what the caller sent on a failed call. The case keeps its intent and is strengthened, asserting the failed status, the value-free CLI_USAGE code, that the name comes back on no channel, and that the reply conforms to every advertised tool's declared schema. No dependency was added or moved BY THIS BRANCH. `@cosyte/deid` in optionalDependencies and the `@cosyte/hl7` range arrive from main, unmodified.
Contributor
Author
Rebased onto a moved base, and what it means for the manifest check
The manifest check. The acceptance criterion for "no dependency was added" is graded as which is clean, exit 0. The PR diff (11 files, all under Re-verified on the merged tree
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Spec:
work/specs/S0089-cli-agent-3/spec.md(umbrella).What changed
An MCP client calling a cosyte tool received
structuredContent: { exit, ok }with no schema to checkit against, and a text content block holding the command's stdout, which was a different value from
the structured one. An agent had to pattern-match a text blob to learn whether it was holding data or
a diagnostic.
outputSchema(and atitle), withtype: "object"at theroot as the protocol requires.
McpToolDefgained both as required fields, so a tool cannot beadded without one.
negative verdict, hard failure, usage error, internal error. The result carries
ok,status(
success/verdict/failed),exit,codeon a failure, and the tool's own payload underdata. The payload an agent got before is still there: the parsed model and warnings (or a recordstream, for a multi-record input), the validation verdict and findings, the value-free structural
summary, the converted Bundle.
sees exactly what a schema-aware client validates.
src/mcp/server.tswere widened. They are explicit field-by-field allow-lists,so a schema or a payload added in
tools.tsalone would never have reached the wire:tools/listnow copies the title and output schema, and
tools/callpasses the whole structured value through.docs-content/mcp.mdand the README's MCP section document the published schema and the serializedtext block; a changeset and a
CHANGELOG.mdentry are included.PHI posture
The
structuredContentchannel is now wide enough to carry a value, so it is closed by constructionrather than by review: on a failed call every property is drawn from a fixed set (the outcome
vocabulary, the exit-code contract, the
CLI_CODESregistry). The diagnostic code is matched againstthe registry rather than lifted out of
stderr, so a future value-bearing diagnostic still cannotleak through it.
One deliberate behaviour change follows: an unknown tool name is no longer echoed back. The old
text block said
unknown tool '<name>', and a tool name is caller-supplied text like any otherargument. The same applies to
convert's rejected--totarget.The PHI-leak matrix now covers the agent surface: for
parse/convertthe payload is the datachannel and every other property is asserted value-free; for
validate/inspectno sentinel mayappear anywhere in the result; and every failure mode over PHI-laden input is value-free whole.
How conformance is checked, with no new dependency
No runtime, optional or development dependency was added or moved.
test/helpers/schema-conformance.tsis a dependency-free checker over exactly the JSON-Schema keyword subset these schemas use, and it
throws on any keyword or type it does not implement, so it can never report green over a schema it
did not understand. A negative control proves it says no to a corrupted value one way per declared
constraint, and its violation messages name the path and the expectation, never the value.
Independently,
test/mcp-server.test.tscallslistTools()beforecallTool(), which is what makesthe SDK client compile and run its own validator against each advertised schema. That was confirmed
non-vacuous by temporarily injecting an undeclared property, which the client rejected.
Verification
pnpm typecheckpass,pnpm lintpass,pnpm format:checkpasspnpm test:coverage33 files / 551 tests pass; statements 98.01, branches 93.13, functions 94.35,lines 98.68, all per-directory gates intact
pnpm smoke,pnpm phi-scan,pnpm check:no-emdash,pnpm check:no-internal-refsall passgit diff --exit-code <base> -- package.json test/exit-code-matrix.test.ts pnpm-lock.yamlclean