fix: the fleet-tools test depended on a private SDK field, so it failed on file order - #297
fix: the fleet-tools test depended on a private SDK field, so it failed on file order#297saucam wants to merge 1 commit into
Conversation
…ed on file order `daemon-check` failed on #295 with a TypeError, not an assertion mismatch: undefined is not an object (evaluating 'Object.keys(server.instance._registeredTools)') at src/tests/collaboration.test.ts:2374 `_registeredTools` is a PRIVATE field of the MCP SDK's McpServer. Two ways that read goes wrong, and the second is what bit us: 1. It breaks on any SDK upgrade that renames or restructures internals. 2. `src/tests/provider-claude.test.ts` installs a process-global `mock.module("@anthropic-ai/claude-agent-sdk", ...)` whose fake `createSdkMcpServer` returns `{ instance: { tools } }` — no `_registeredTools`. Bun's module mocks leak across files within a run, so whether this test saw the real SDK or that fake depended entirely on which file the runner reached first. Locally collaboration.test.ts is 4th and provider-claude.test.ts is 14th, so it passed; CI's readdir order differs and it threw. Neither #295 nor #294 caused this — #295 touches no fleet code, #294 touches no fleet.ts, and the test passes on main locally. It is a latent order-dependent bug that surfaces whenever file ordering shifts, which adding any test file can do. Fix the coupling rather than the symptom. `buildFleetMcpServer` now builds its tool list through an exported `fleetToolDefinitions()` and passes that array through verbatim; the test asserts on those definitions instead of reaching into the SDK. That still proves what the old test set out to prove — that `pick()` actually filters what reaches the model, not just that the constant is well-formed — while being immune to both the SDK's internals and the mock leak. Adds a second test pinning the seam the split introduces: `fleetToolDefinitions` would be worthless if the builder stopped using it. That one asserts only the SDK's PUBLIC config shape, so it holds under a real or a mocked SDK. Verified under both file orders (provider-claude first and last) and on the full suite: 2311 pass, 0 fail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Redundant — #296 landed the same fix (a0bc9a2) about an hour before I started, and my local #296's approach is also the better one. It derives Closing. One observation from my diagnosis that may still be worth a follow-up: |
There was a problem hiding this comment.
🔮 Oracle Review
🎯 Start Here
src/daemon/fleet.ts (~15 min) — Logic changes in fleet.ts
📋 PR Summary
What this PR does: Fixes flaky test failures in collaboration.test.ts by removing dependency on a private SDK field (_registeredTools) that was causing failures when Bun's module mocks leaked between test files in different execution orders.
Key changes:
- Extracted fleetToolDefinitions() as a public API function in fleet.ts
- Refactored buildFleetMcpServer to use the exported tool definitions instead of accessing private SDK internals
- Updated tests to assert on public fleetToolDefinitions() instead of SDK's _registeredTools field
- Added integration test verifying buildFleetMcpServer correctly calls fleetToolDefinitions()
Areas affected: src/daemon/fleet.ts, src/tests/collaboration.test.ts
Testing notes: Verified 227+ tests pass in both problematic and reverse file orderings; full suite 2311 tests pass with lint and typecheck clean.
🔍 Code Review
This is an excellent root-cause fix that properly decouples tests from private SDK internals rather than patching the symptom. The author's thorough investigation identified the Bun module mock leakage mechanism and implemented a clean separation that makes tests robust against SDK changes and execution order.
What's good:
- ✨ Strong diagnostic work identifying the coupling to private SDK fields as the root cause
- ✨ Proper architectural fix extracting a stable public API (fleetToolDefinitions) for testing seams
- ✨ Comprehensive verification across multiple test execution orderings to prevent regression
- ✨ Clear documentation of the issue mechanism and why adjacent PRs were not responsible
Generated by Oracle - Highflame's AI Code Reviewer
| @@ -559,10 +589,7 @@ export function buildFleetMcpServer( | |||
| const pick = <T extends { name: string }>(tools: T[]): T[] => | |||
There was a problem hiding this comment.
Avoid as never type assertion
The refactor introduces an as never cast on the tools argument passed to createSdkMcpServer. While this may be necessary to satisfy the compiler after extracting buildFleetTools, it completely bypasses type checking for the tool definitions. If the tool shape returned by buildFleetTools deviates from what the SDK expects, this error will be hidden until runtime. Consider using satisfies or explicit generic typing if possible to maintain type safety.
Suggested fix:
| const pick = <T extends { name: string }>(tools: T[]): T[] => | |
| If strict typing is impossible, consider a comment explaining *why* `as never` is required, or check if the SDK's `createSdkMcpServer` can be typed to accept the inferred return type of `buildFleetTools`. |
Unblocks #295, whose
daemon-checkis red. The failure is not #295's fault — it touches no fleet code.What actually failed
Not an assertion mismatch — a
TypeError:_registeredToolsis a private field of the MCP SDK'sMcpServer. Reading it goes wrong two ways, and the second is what bit us:src/tests/provider-claude.test.ts:56installs a process-globalmock.module("@anthropic-ai/claude-agent-sdk", ...)whose fakecreateSdkMcpServerreturns{ instance: { tools } }— no_registeredTools. Bun's module mocks leak across files within a run, so whether this test saw the real SDK or that fake depended entirely on which file the runner reached first.Locally
collaboration.test.tsis 4th andprovider-claude.test.tsis 14th, so it passes. CI's readdir order differs, and it threw.Why it's not #294 or #295
config.ts,daemon/guard/*,session.ts, docs — no fleet code, no dependency filesfleet.tsmainlocally, and both SDKs are lockfile-pinned (claude-agent-sdk@0.3.220,@modelcontextprotocol/sdk@1.29.0), identical local and CIIt's a latent order-dependent bug that surfaces whenever file ordering shifts — which adding any test file can do. #295 adds
src/daemon/guard/repeat-tool.test.ts.I confirmed the leak mechanism directly: two synthetic files in the same run, one registering that mock and one importing the SDK. Alone the second sees
_registeredResources,…,_registeredTools,…; after the mock it seestools— exactly the CI shape.The fix
Fix the coupling, not the symptom.
buildFleetMcpServernow builds through an exportedfleetToolDefinitions()and passes that array through verbatim; the test asserts on those definitions instead of reaching into the SDK.That still proves what the old test set out to prove — that
pick()actually filters what reaches the model, not merely that the constant is well-formed — while being immune to both the SDK's internals and the mock leak.Adds a second test pinning the seam the split introduces:
fleetToolDefinitionswould be worthless if the builder stopped calling it. That one asserts only the SDK's public config shape, so it holds under a real or mocked SDK.Verification
provider-claudefirst) — 227 pass, 0 failNote for #295
Merge
maininto it after this lands and itsdaemon-checkshould go green.Worth knowing separately
provider-claude.test.ts's mock is still process-global and still shape-unfaithful. Nothing else reads_registeredToolstoday (I checked — this test was the only consumer), so this PR closes the live issue. But the next thing to reach into a mocked SDK object hits the same trap. Making that mock mirror the real shape would be a reasonable follow-up; I left it out to keep this fix to the failing path.