Skip to content

S0089-cli-agent-3: publish an output schema per MCP tool and conform every result - #49

Merged
NSchatz merged 2 commits into
mainfrom
sdd/S0089-cli-agent-3
Aug 29, 2026
Merged

S0089-cli-agent-3: publish an output schema per MCP tool and conform every result#49
NSchatz merged 2 commits into
mainfrom
sdd/S0089-cli-agent-3

Conversation

@NSchatz

@NSchatz NSchatz commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Spec: work/specs/S0089-cli-agent-3/spec.md (umbrella).

What changed

An MCP client calling a cosyte tool received structuredContent: { exit, ok } with no schema to check
it against, and a text content block holding the command's stdout, which was a different value from
the structured one. An agent had to pattern-match a text blob to learn whether it was holding data or
a diagnostic.

  • Every advertised tool declares an outputSchema (and a title), with type: "object" at the
    root as the protocol requires. McpToolDef gained both as required fields, so a tool cannot be
    added without one.
  • Every dispatch path returns structured content conforming to its own tool's schema: success,
    negative verdict, hard failure, usage error, internal error. The result carries ok, status
    (success / verdict / failed), exit, code on a failure, and the tool's own payload under
    data. The payload an agent got before is still there: the parsed model and warnings (or a record
    stream, for a multi-record input), the validation verdict and findings, the value-free structural
    summary, the converted Bundle.
  • The text content block is the serialized JSON of that structured value, so a text-only client
    sees exactly what a schema-aware client validates.
  • Both handlers in src/mcp/server.ts were widened. They are explicit field-by-field allow-lists,
    so a schema or a payload added in tools.ts alone would never have reached the wire: tools/list
    now copies the title and output schema, and tools/call passes the whole structured value through.
  • docs-content/mcp.md and the README's MCP section document the published schema and the serialized
    text block; a changeset and a CHANGELOG.md entry are included.

PHI posture

The structuredContent channel is now wide enough to carry a value, so it is closed by construction
rather than by review: on a failed call every property is drawn from a fixed set (the outcome
vocabulary, the exit-code contract, the CLI_CODES registry). The diagnostic code is matched against
the registry rather than lifted out of stderr, so a future value-bearing diagnostic still cannot
leak through it.

One deliberate behaviour change follows: an unknown tool name is no longer echoed back. The old
text block said unknown tool '<name>', and a tool name is caller-supplied text like any other
argument. The same applies to convert's rejected --to target.

The PHI-leak matrix now covers the agent surface: for parse / convert the payload is the data
channel and every other property is asserted value-free; for validate / inspect no sentinel may
appear anywhere in the result; and every failure mode over PHI-laden input is value-free whole.

How conformance is checked, with no new dependency

No runtime, optional or development dependency was added or moved. test/helpers/schema-conformance.ts
is a dependency-free checker over exactly the JSON-Schema keyword subset these schemas use, and it
throws on any keyword or type it does not implement, so it can never report green over a schema it
did not understand. A negative control proves it says no to a corrupted value one way per declared
constraint, and its violation messages name the path and the expectation, never the value.

Independently, test/mcp-server.test.ts calls listTools() before callTool(), which is what makes
the SDK client compile and run its own validator against each advertised schema. That was confirmed
non-vacuous by temporarily injecting an undeclared property, which the client rejected.

Verification

  • pnpm typecheck pass, pnpm lint pass, pnpm format:check pass
  • pnpm test:coverage 33 files / 551 tests pass; statements 98.01, branches 93.13, functions 94.35,
    lines 98.68, all per-directory gates intact
  • pnpm smoke, pnpm phi-scan, pnpm check:no-emdash, pnpm check:no-internal-refs all pass
  • git diff --exit-code <base> -- package.json test/exit-code-matrix.test.ts pnpm-lock.yaml clean

…every result

Every advertised tool now declares an `outputSchema` (and a `title`), every
dispatch path returns structured content conforming to the schema its own tool
declared, and the text content block is the serialized JSON of that same
structured value.

The structured result carries `ok`, a `status` of success / verdict / failed,
the `exit` code, a stable `code` on a failed call, and the tool's own payload
under `data` (the parsed model and warnings, or a record stream for a
multi-record input; the validation verdict and findings; the value-free
structural summary; the converted Bundle). `status` is the declared property
that separates a negative verdict about the message from a call that produced
nothing, which the old `{ exit, ok }` pair could not express.

Both handlers in `src/mcp/server.ts` are explicit allow-lists, so both were
widened: `tools/list` now copies the title and the output schema, and
`tools/call` passes the whole structured value through instead of rebuilding
`{ exit, ok }`.

PHI posture: on a failed call every property of the structured result is drawn
from a fixed set (the outcome vocabulary, the exit-code contract, the
CLI_CODES registry), so no part of the caller's input can appear in it. The
diagnostic code is matched against the registry rather than lifted out of
stderr, and an unknown tool name is no longer echoed back, because a tool name
is caller-supplied text like any other argument.

Test coverage validates each emitted structured value against its own tool's
declared schema using a dependency-free checker (test/helpers) that refuses any
schema keyword it does not implement, with a negative control proving it can
fail; the text-block round trip; the advertised-schema assertions over the SDK
client (which validates the replies itself once tools/list has been called);
and the unhappy paths. The PHI-leak matrix now covers the agent surface.

No dependency was added or moved. Tool names, tool count, tools/list ordering,
input schemas, the exit-code contract and the verdict-vs-isError rule are
unchanged.
…his branch)

`cli` main advanced from 1f26d41 to 9a17fed while this branch was open, adding
the de-identification work. Three files conflicted.

CHANGELOG.md and test/phi-leak.test.ts: both sides kept, they are additive.

test/mcp-tools.test.ts: main's two new cases are kept, and one assertion in
"`redact` is an unknown tool name, answered value-free" is updated rather than
kept verbatim. It asserted the text block contains `unknown tool '<name>'`, and
this branch deliberately stops echoing a caller-supplied tool name: the text
block is now the serialization of the structured result, and the structured
result carries no part of what the caller sent on a failed call. The case keeps
its intent and is strengthened, asserting the failed status, the value-free
CLI_USAGE code, that the name comes back on no channel, and that the reply
conforms to every advertised tool's declared schema.

No dependency was added or moved BY THIS BRANCH. `@cosyte/deid` in
optionalDependencies and the `@cosyte/hl7` range arrive from main, unmodified.
@NSchatz

NSchatz commented Aug 29, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto a moved base, and what it means for the manifest check

cli main advanced from 1f26d41 (the pin this work was specified against) to 9a17fed while this
branch was open, landing the de-identification work. origin/main is merged in as 5450368. Three
files conflicted:

  • CHANGELOG.md and test/phi-leak.test.ts: both sides kept, they are additive.
  • test/mcp-tools.test.ts: main's two new cases are kept, and one assertion inside
    "redact is an unknown tool name, answered value-free" is updated rather than kept verbatim. It
    asserted content[0].text contains unknown tool '<name>'. This branch deliberately stops echoing
    a caller-supplied tool name, and that follows from the contract rather than from taste: the text
    block is now the serialization of the structured result, and the structured result carries no part
    of what the caller sent on a failed call. The case keeps its intent and is strengthened, asserting
    the failed status, the value-free CLI_USAGE code, that the name comes back on no channel at all,
    and that the reply conforms to every advertised tool's declared schema.

The manifest check. The acceptance criterion for "no dependency was added" is graded as
git diff --exit-code 1f26d41f45123d0879afa5ef86038a20219c2d0c -- package.json. Run today that shows
one line, @cosyte/deid in optionalDependencies, which arrives from main and not from this
branch. The measurement that actually answers the criterion is against the real base:

git diff --exit-code origin/main...HEAD -- package.json pnpm-lock.yaml test/exit-code-matrix.test.ts

which is clean, exit 0. The PR diff (11 files, all under src/mcp/, test/, docs-content/mcp.md,
README.md, CHANGELOG.md, .changeset/) touches no manifest and no exit-code file.

Re-verified on the merged tree

  • pnpm typecheck pass, pnpm lint pass, pnpm format:check pass
  • pnpm test:coverage 37 files / 668 tests pass; statements 98.22, branches 93.50, functions 95.26,
    lines 98.82, per-directory gates intact
  • pnpm check:agent-notes, pnpm check:no-emdash, pnpm check:no-internal-refs, pnpm phi-scan,
    pnpm smoke all pass

@NSchatz
NSchatz merged commit 380d100 into main Aug 29, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant