fix(llm): treat a null tool argument as absent for optional Zod fields - #2308
Open
yisding wants to merge 1 commit into
Open
fix(llm): treat a null tool argument as absent for optional Zod fields#2308yisding wants to merge 1 commit into
yisding wants to merge 1 commit into
Conversation
`zodSchemaToJsonSchema` targets `openAi`, which rewrites an `.optional()` property into a required nullable one because a strict tool schema must list every property in `required`. That leaves null as the only way for a model to say "not provided", but tool arguments are validated against the original Zod schema, where `.optional()` accepts undefined and rejects null. Any tool with a bare `.optional()` field fails with `Arguments parsing failed` as soon as the model fills the key in with null. Defaulted fields already round-tripped through the null sentinel resolved in `injectSchemaDefaults` (livekit#2150). Extend the same inverse mapping to optional fields: drop a null when the property is absent from `required` and its schema does not allow null, so Zod sees undefined. Defaults still win because the drop runs after recursion, and a property that is required or genuinely nullable keeps its null so real contract violations still surface. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: 57a9479 The changes in this PR will be included in the next version bump. This PR includes changesets to release 39 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
A function tool whose Zod schema has a bare
.optional()field fails withArguments parsing failedas soon as the model sendsnullfor that field. The schema we hand the model and the schema we validate against disagree about how to spell "absent".zodSchemaToJsonSchemausestarget: 'openAi'(zod-utils.ts:126), whoseforceOptionalIntoNullablerewrites an.optional()property into a required, nullable one — necessarily, because a strict tool schema must list every property inrequired. Sonullbecomes the model's only way to say "not provided". ButexecuteToolCallvalidates against the original Zod schema (utils.ts:612), where.optional()acceptsundefinedand rejectsnull.Minimal repro:
This is not specific to
strictToolSchema. The emitted schema for a bare.optional()field is byte-identical withstricton and off —forceOptionalIntoNullableruns off the target, not the strict flag — and validation never consults the flag. TurningstrictToolSchemaon just makes it deterministic rather than occasional, because the decoder then enforcesrequiredand the model must fill the key in.The docs already advise "prefer
.nullable()over.optional()" for OpenAI strict mode. The problem is that ignoring it fails silently: no error at schema build, no warning, no test-time signal — just a runtime tool failure on a live call. For contrast, OpenAI's own SDK treats this exact shape as unsupported and throws at schema construction (openai/_vendor/zod-to-json-schema/parsers/object.js), on precisely the predicateisOptional() && !isNullable() && no defaultValue.#2150 already established that null is the wire encoding for "absent" and added the inverse mapping in
injectSchemaDefaults— but only for fields with a.default(). Optional fields get the identical wire encoding with no return path. This extends that same mechanism to cover them.Changes Made
injectSchemaDefaultsnow drops anullproperty value when the property is absent from the parent'srequiredlist and its schema does not allow null, so Zod seesundefined— which is what.optional()accepts..nullish()regression guard that must keep its null.Three properties keep it narrow:
jsonSchemaAllowsNullhelper, so.nullish()and.nullable()are untouched.requirednon-nullable field keeps its null and still errors — the existingshould reject null for required arguments without defaultstest still passes unchanged.No public API change, and no change to schema generation — only the inverse mapping at validation time.
requiredis read off thejsonSchema7-target schemainjectSchemaDefaultsalready receives, where it still reflects true optionality.Pre-Review Checklist
Testing
Automated tests added/updated
All tests pass
pnpm exec vitest run agents— 2140 passed, 5 skipped, 0 failedConfirmed the new tests fail without the fix (3 of the 4 fail when the
utils.tschange is stashed; the.nullish()guard passes both ways by design)tsc --noEmit— cleanprettier --check— cleaneslint— no new findings (one pre-existingno-explicit-anywarning inutils.ts, present atmaintoo)api:check(API Extractor) — passes, no API surface changeAdditional Notes
An alternative fix would be to throw at schema-generation time the way OpenAI's SDK does. I didn't take that route: it would break every existing user who has a bare
.optional()in a tool schema and is currently getting away with it because their model omits the key rather than nulling it. Resolving the sentinel is backward compatible, and it matches the direction #2150 already set. A dev-time warning could be added on top if you'd want the sharper signal.🤖 Generated with Claude Code