Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 25 additions & 1 deletion packages/mcp/DIRECTORY_PREFLIGHT.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,27 @@ ChatGPT directory.
- [ ] Run `npm run test --workspace @terminal49/mcp -- --run`,
`npm run build --workspace @terminal49/mcp`, and
`npm run lint --workspace @terminal49/mcp`.
- [ ] Confirm every tool has a non-empty `title`, `readOnlyHint`, and
`destructiveHint`. Confirm `track_container` is the only write tool and
no tool is destructive. CI enforces this in
[`annotations.test.ts`](./src/annotations.test.ts).
- [ ] Confirm tool names remain at most 64 characters, list `page_size` stays
bounded to 1–25, and the advertised `include` choices/defaults remain
lean. CI locks these contracts in
[`protocol-compat.test.ts`](./src/protocol-compat.test.ts).
- [ ] Compare the live `tools/list` response with the tool reference in
`docs/mcp/home.mdx`. Confirm parameter names, required fields, defaults,
limits, and descriptions match.
- [ ] Confirm the live connector still uses HTTPS, OAuth, and Streamable HTTP
at `https://mcp.terminal49.com`. CI checks the locked submission metadata
and exercises the protocol surface over Streamable HTTP in the annotation
and protocol-compatibility tests.
- [ ] Confirm the tool surface stays task-specific with no catch-all request
tool such as `api_request`.
- [ ] Confirm tools and server instructions request only their declared
parameters—not chat history, conversation context, or user memory. CI
rejects conversation- or memory-shaped arguments and instructions in
[`protocol-compat.test.ts`](./src/protocol-compat.test.ts).
- [ ] Inspect successful tool results and server/tool instructions. They must
not contain `_agent_steering`, `_response_contract`,
`presentation_guidance`, `suggested_follow_ups`, or `suggested_tools`.
Expand All @@ -18,7 +36,13 @@ ChatGPT directory.
- [ ] Confirm `track_container.number` is required and `get_container`
`transport_events` returns only event counts plus the latest event. Use
`get_container_transport_events` for full history.
- [ ] Run `mcpdemo` against `https://mcp.terminal49.com` using Anthropic's
- [ ] Recheck the [privacy policy](https://terminal49.com/privacy) and MCP data
practices: tools access the authenticated private account, use only tool
arguments, and only `track_container` creates data.
- [ ] Populate and run `mcpdemo` with at least three working prompts. Exercise
every tool in MCP Inspector and a Claude custom connector; use the
[live eval](./eval/README.md) as the automated baseline.
- [ ] Review the results against Anthropic's
current [submission requirements](https://claude.com/docs/connectors/building/submission)
and [Software Directory Policy](https://support.claude.com/en/articles/13145358-anthropic-software-directory-policy).
Resolve every failure before resubmitting.
20 changes: 20 additions & 0 deletions packages/mcp/src/annotations.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,19 @@ describe('MCP tool annotations', () => {
expect(annotations?.idempotentHint).toBe(false);
});

it('advertises track_container as the only write and no destructive tools', () => {
const tools = getRegisteredTools();
const writeTools = Object.entries(tools)
.filter(([, tool]) => tool.annotations?.readOnlyHint === false)
.map(([name]) => name);
const destructiveTools = Object.entries(tools)
.filter(([, tool]) => tool.annotations?.destructiveHint === true)
.map(([name]) => name);

expect(writeTools).toEqual(['track_container']);
expect(destructiveTools).toEqual([]);
});

it('marks every tool as private-account-only and non-destructive', () => {
const tools = getRegisteredTools();

Expand Down Expand Up @@ -150,6 +163,13 @@ describe('MCP tool annotations', () => {
expect(tool.title, `${name}.title`).toEqual(expect.any(String));
expect(tool.title?.trim().length, `${name}.title`).toBeGreaterThan(0);
expect(tool.annotations, name).toBeDefined();
expect(tool.annotations?.readOnlyHint, `${name}.readOnlyHint`).toBeTypeOf(
'boolean',
);
expect(
tool.annotations?.destructiveHint,
`${name}.destructiveHint`,
).toBeTypeOf('boolean');
}
});

Expand Down
69 changes: 66 additions & 3 deletions packages/mcp/src/protocol-compat.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,10 @@ const openConnections: Array<{

type AdvertisedProperty = {
type?: string;
default?: unknown;
maxLength?: number;
maxItems?: number;
exclusiveMinimum?: number;
maximum?: number;
description?: string;
items?: { enum?: string[] };
Expand Down Expand Up @@ -173,11 +176,35 @@ describe('MCP protocol compatibility', () => {

expect(tools).toHaveLength(10);
for (const tool of tools) {
expect(tool.name.length, tool.name).toBeLessThanOrEqual(64);
expect(tool.inputSchema.properties, tool.name).not.toHaveProperty(
'intent',
);
for (const propertyName of Object.keys(
tool.inputSchema.properties ?? {},
)) {
expect(propertyName, `${tool.name}.${propertyName}`).not.toMatch(
/(?:chat|conversation|history|memory|messages?)/i,
);
}
}

const advertisedInstructions = [
TERMINAL49_SERVER_INSTRUCTIONS,
...tools.flatMap((tool) => [
tool.description ?? '',
...Object.values(
(tool.inputSchema as AdvertisedInputSchema).properties ?? {},
).map((property) => property.description ?? ''),
]),
].join('\n');
expect(advertisedInstructions).not.toMatch(
/\b(?:send|provide|share|upload|attach)\b.{0,40}\b(?:chat|conversation) history\b/i,
);
Comment on lines +201 to +203

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Check for requests to provide conversation context

If a tool or server description asks callers to “provide conversation context,” both assertions pass because they only recognize chat/conversation history or user memory. This contradicts the new preflight claim that CI rejects conversation-context requests and could allow directory-disallowed context collection to reach submission; extend the guard to cover conversation context and equivalent phrasing.

Useful? React with 👍 / 👎.

expect(advertisedInstructions).not.toMatch(
/\b(?:send|provide|share|upload|attach)\b.{0,40}\buser memory\b/i,
);

expect(
toolSchemas.get('search_container')?.properties?.query,
).toMatchObject({
Expand All @@ -195,11 +222,47 @@ describe('MCP protocol compatibility', () => {
'list_containers',
'list_tracking_requests',
]) {
expect(toolSchemas.get(name)?.properties?.page_size?.maximum, name).toBe(
25,
);
expect(
toolSchemas.get(name)?.properties?.page,
`${name}.page`,
).toMatchObject({
type: 'integer',
exclusiveMinimum: 0,
});
expect(
toolSchemas.get(name)?.properties?.page_size,
`${name}.page_size`,
).toMatchObject({
default: 25,
type: 'integer',
exclusiveMinimum: 0,
maximum: 25,
});
}

expect(toolSchemas.get('get_container')?.properties?.include).toMatchObject(
{
default: ['shipment'],
items: {
enum: ['shipment', 'pod_terminal', 'transport_events'],
},
},
);
expect(
toolSchemas.get('list_containers')?.properties?.include,
).toMatchObject({
items: {
enum: ['shipment', 'pod_terminal'],
},
maxItems: 2,
});
expect(
toolSchemas.get('get_shipment_details')?.properties?.include_containers,
).toMatchObject({ default: true });
expect(
toolSchemas.get('list_shipments')?.properties?.include_containers,
).toMatchObject({ default: false });

const trackingRequestSchema = toolSchemas.get('list_tracking_requests');
expect(trackingRequestSchema?.properties).not.toHaveProperty('filters');
expect(trackingRequestSchema?.properties).not.toHaveProperty(
Expand Down
Loading