Skip to content

fix(llm): treat a null tool argument as absent for optional Zod fields - #2308

Open
yisding wants to merge 1 commit into
livekit:mainfrom
yisding:fix/optional-null-sentinel
Open

fix(llm): treat a null tool argument as absent for optional Zod fields#2308
yisding wants to merge 1 commit into
livekit:mainfrom
yisding:fix/optional-null-sentinel

Conversation

@yisding

@yisding yisding commented Aug 19, 2026

Copy link
Copy Markdown

Description

A function tool whose Zod schema has a bare .optional() field fails with Arguments parsing failed as soon as the model sends null for that field. The schema we hand the model and the schema we validate against disagree about how to spell "absent".

zodSchemaToJsonSchema uses target: 'openAi' (zod-utils.ts:126), whose forceOptionalIntoNullable rewrites an .optional() property into a required, nullable one — necessarily, because a strict tool schema must list every property in required. So null becomes the model's only way to say "not provided". But executeToolCall validates against the original Zod schema (utils.ts:612), where .optional() accepts undefined and rejects null.

Minimal repro:

const t = tool({
  name: 'search',
  description: 'search',
  parameters: z.object({ query: z.string(), limit: z.number().optional() }),
  execute: async (args) => args,
});
// args: '{"query":"cats","limit":null}'
// -> Arguments parsing failed: [{ expected: 'number', received: 'null', path: ['limit'] }]

This is not specific to strictToolSchema. The emitted schema for a bare .optional() field is byte-identical with strict on and off — forceOptionalIntoNullable runs off the target, not the strict flag — and validation never consults the flag. Turning strictToolSchema on just makes it deterministic rather than occasional, because the decoder then enforces required and the model must fill the key in.

The docs already advise "prefer .nullable() over .optional()" for OpenAI strict mode. The problem is that ignoring it fails silently: no error at schema build, no warning, no test-time signal — just a runtime tool failure on a live call. For contrast, OpenAI's own SDK treats this exact shape as unsupported and throws at schema construction (openai/_vendor/zod-to-json-schema/parsers/object.js), on precisely the predicate isOptional() && !isNullable() && no defaultValue.

#2150 already established that null is the wire encoding for "absent" and added the inverse mapping in injectSchemaDefaults — but only for fields with a .default(). Optional fields get the identical wire encoding with no return path. This extends that same mechanism to cover them.

Changes Made

  • injectSchemaDefaults now drops a null property value when the property is absent from the parent's required list and its schema does not allow null, so Zod sees undefined — which is what .optional() accepts.
  • Tests for the flat, constrained, and nested (object + array element) cases, plus a .nullish() regression guard that must keep its null.
  • Patch changeset.

Three properties keep it narrow:

  • Defaults still win. The drop test runs on the result of the recursive call, so the existing default branch has already substituted the default and a defaulted field never reaches the drop.
  • Genuine nulls survive. Gated on the existing jsonSchemaAllowsNull helper, so .nullish() and .nullable() are untouched.
  • Real violations still fail. A required non-nullable field keeps its null and still errors — the existing should reject null for required arguments without defaults test still passes unchanged.

No public API change, and no change to schema generation — only the inverse mapping at validation time. required is read off the jsonSchema7-target schema injectSchemaDefaults already receives, where it still reflects true optionality.

Pre-Review Checklist

  • Build passes: All builds (lint, typecheck, tests) pass locally
  • AI-generated code reviewed: Removed unnecessary comments and ensured code quality
  • Changes explained: All changes are properly documented and justified above
  • Scope appropriate: All changes relate to the PR title
  • Video demo: n/a — no runtime/UX surface, covered by unit tests

Testing

  • Automated tests added/updated

  • All tests pass

  • pnpm exec vitest run agents2140 passed, 5 skipped, 0 failed

  • Confirmed the new tests fail without the fix (3 of the 4 fail when the utils.ts change is stashed; the .nullish() guard passes both ways by design)

  • tsc --noEmit — clean

  • prettier --check — clean

  • eslint — no new findings (one pre-existing no-explicit-any warning in utils.ts, present at main too)

  • api:check (API Extractor) — passes, no API surface change

Additional Notes

An alternative fix would be to throw at schema-generation time the way OpenAI's SDK does. I didn't take that route: it would break every existing user who has a bare .optional() in a tool schema and is currently getting away with it because their model omits the key rather than nulling it. Resolving the sentinel is backward compatible, and it matches the direction #2150 already set. A dev-time warning could be added on top if you'd want the sharper signal.


🤖 Generated with Claude Code

`zodSchemaToJsonSchema` targets `openAi`, which rewrites an `.optional()`
property into a required nullable one because a strict tool schema must list
every property in `required`. That leaves null as the only way for a model to
say "not provided", but tool arguments are validated against the original Zod
schema, where `.optional()` accepts undefined and rejects null. Any tool with a
bare `.optional()` field fails with `Arguments parsing failed` as soon as the
model fills the key in with null.

Defaulted fields already round-tripped through the null sentinel resolved in
`injectSchemaDefaults` (livekit#2150). Extend the same inverse mapping to optional
fields: drop a null when the property is absent from `required` and its schema
does not allow null, so Zod sees undefined. Defaults still win because the drop
runs after recursion, and a property that is required or genuinely nullable
keeps its null so real contract violations still surface.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@yisding
yisding requested a review from a team as a code owner August 19, 2026 21:58
@changeset-bot

changeset-bot Bot commented Aug 19, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 57a9479

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 39 packages
Name Type
@livekit/agents Patch
@livekit/agents-plugin-anam Patch
@livekit/agents-plugin-anthropic Patch
@livekit/agents-plugin-assemblyai Patch
@livekit/agents-plugin-azure Patch
@livekit/agents-plugin-baseten Patch
@livekit/agents-plugin-bey Patch
@livekit/agents-plugin-cartesia Patch
@livekit/agents-plugin-cerebras Patch
@livekit/agents-plugin-deepgram Patch
@livekit/agents-plugin-did Patch
@livekit/agents-plugin-elevenlabs Patch
@livekit/agents-plugin-fishaudio Patch
@livekit/agents-plugin-google Patch
@livekit/agents-plugin-hedra Patch
@livekit/agents-plugin-hume Patch
@livekit/agents-plugin-inworld Patch
@livekit/agents-plugin-krisp Patch
@livekit/agents-plugin-lemonslice Patch
@livekit/agents-plugin-liveavatar Patch
@livekit/agents-plugin-livekit Patch
@livekit/agents-plugin-minimax Patch
@livekit/agents-plugin-mistral Patch
@livekit/agents-plugin-mistralai Patch
@livekit/agents-plugin-neuphonic Patch
@livekit/agents-plugin-openai Patch
@livekit/agents-plugin-perplexity Patch
@livekit/agents-plugin-phonic Patch
@livekit/agents-plugin-protoface Patch
@livekit/agents-plugin-resemble Patch
@livekit/agents-plugin-rime Patch
@livekit/agents-plugin-runway Patch
@livekit/agents-plugin-sarvam Patch
@livekit/agents-plugin-silero Patch
@livekit/agents-plugin-soniox Patch
@livekit/agents-plugin-tavus Patch
@livekit/agents-plugins-test Patch
@livekit/agents-plugin-trugen Patch
@livekit/agents-plugin-xai Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@CLAassistant

CLAassistant commented Aug 19, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.

Open in Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants