diff --git a/apps/website/content/docs/chat/guides/client-tools.mdx b/apps/website/content/docs/chat/guides/client-tools.mdx index cd5d56bde..8b643a53d 100644 --- a/apps/website/content/docs/chat/guides/client-tools.mdx +++ b/apps/website/content/docs/chat/guides/client-tools.mdx @@ -108,6 +108,93 @@ async function moveStop(args: ToolArgs) { const move = action('Move a stop to another day.', moveSchema, moveStop); ``` +## Terminal tools with `followUp: false` + +By default, resolving a client tool starts a new run so the model can react to the result. Pass `followUp: false` when a tool **ends the turn** — a summary card, a confirmation receipt, anything the model has nothing further to say about. The result is still recorded on the server; the model simply is not asked to respond to it. + +```typescript +const clientTools = tools({ + trip_summary: view( + 'Show a final trip summary card. Call this last — it ends the turn.', + z.object({ title: z.string(), days: z.array(z.string()) }), + TripSummaryCardComponent, + { followUp: false }, + ), +}); +``` + +Follow-up is decided **per tool-call group**, not per tool. If the model calls three tools in one turn and any one of them wants a follow-up, the whole group continues in a single run once every result has settled. Only when *every* tool in the group is terminal does the turn end. + + + A terminal group has no follow-up run to carry its results, so the adapter writes them to the server directly. On `@threadplane/langgraph` that uses the transport's `updateState`. A **custom transport that does not implement `updateState`** falls back to attaching the results to the user's *next* message — correct in the normal path, but the results are lost if the page reloads first, leaving a tool call with no result and a provider error on the next turn. If you supply your own transport and use terminal tools, implement `updateState`. + + +## Re-running tools safely with `idempotent` + +`action()` also accepts `idempotent`. It matters only when you supply a `[clientToolExecutionGuard]` — a durable store that claims each tool call before the browser executes it, so a handler with real side effects cannot run twice across a reload or a reconnect. + +```typescript +const clientTools = tools({ + charge_card: action( + 'Charge the saved payment method.', + z.object({ amountCents: z.number() }), + chargeCard, + // default: claimed before execution, fail-closed if interrupted + ), + fetch_quote: action( + 'Fetch a shipping quote.', + z.object({ zip: z.string() }), + fetchQuote, + { idempotent: true }, // safe to re-run; skips the durable claim + ), +}); +``` + +Tools are treated as non-idempotent by default. Mark a tool `idempotent: true` only when re-running it is genuinely harmless — reads, pure computations, lookups. + + + The guard gives you at-most-once *dispatch*, not exactly-once *effects*. If a handler completes its side effect and the browser dies before recording the result, the guard fails closed and reports the call as interrupted. For true end-to-end idempotency, have the handler pass its own idempotency key to the downstream service. + + +## Stopping and continuation limits + +**Stop cancels cleanly.** Pressing stop while a client tool is running aborts the handler, records a cancelled result so the server never holds an unanswered tool call, and does **not** start a new run. The cancelled call will not re-execute. + +Handlers receive an `AbortSignal` — forward it to `fetch` so in-flight work actually stops: + +```typescript +const search = action( + 'Search the catalog.', + z.object({ query: z.string() }), + async ({ query }, { signal }) => { + const res = await fetch(`/api/search?q=${query}`, { signal }); + return res.json(); + }, +); +``` + +**Runaway loops are capped.** A model that keeps calling client tools is stopped after 10 continuation groups per user turn. Tune it with `[clientToolContinuationPolicy]`: + +```typescript +@Component({ + template: ` + + `, +}) +export class ClientToolsComponent { + protected readonly policy = { + maxTurns: 5, // 0 disables the cap + onLimit: (e) => console.warn('client tool loop stopped', e.toolNames), + }; +} +``` + +When the cap trips, tools that already produced a real result keep it; tools that never ran are recorded with a limit error so the thread stays valid. The run does not continue. + ## Typed agent state Tool handlers and components often read agent state. Pair the registry with a typed `AgentRef` so `agent.state()` / `agent.value()` carry your state shape instead of `Record` — see [Typed state via AgentRef](/docs/langgraph/api/provide-agent#typed-state-via-agentref): @@ -130,18 +217,37 @@ import { tools, action, view, ask, type ViewProps, type ToolArgs, type ClientToolDef, type ClientToolRegistry, + type ClientToolExecutionOptions, type ClientToolContinuationOptions, + type ClientToolContinuationPolicy, type ClientToolContinuationLimitEvent, + type ClientToolExecutionStore, type ClientToolExecutionGuard, } from '@threadplane/chat'; ``` | Export | Purpose | |---|---| -| `action(description, schema, handler)` | Declare a function tool (handler return → result) | -| `view(description, schema, component)` | Declare a render-only component tool (auto-acknowledged) | -| `ask(description, schema, component)` | Declare an interactive component tool (emitted value → result) | +| `action(description, schema, handler, options?)` | Declare a function tool (handler return → result) | +| `view(description, schema, component, options?)` | Declare a render-only component tool (auto-acknowledged) | +| `ask(description, schema, component, options?)` | Declare an interactive component tool (emitted value → result) | | `tools(map)` | Freeze a name-keyed registry for `[clientTools]` | | `ViewProps` | Component input prop bag inferred from a schema | | `ToolArgs` | Handler argument type inferred from a schema | | `ClientToolDef` / `ClientToolRegistry` | The tool-definition union and frozen-registry types | +| `ClientToolContinuationOptions` | `{ followUp? }` — accepted by `view()` and `ask()` | +| `ClientToolExecutionOptions` | `{ followUp?, idempotent? }` — accepted by `action()` | +| `ClientToolContinuationPolicy` | `{ maxTurns?, onLimit? }` for `[clientToolContinuationPolicy]` | +| `ClientToolExecutionStore` / `ClientToolExecutionGuard` | Durable claim store for `[clientToolExecutionGuard]` | + +Component inputs on ``: + +| Input | Purpose | +|---|---| +| `[clientTools]` | The frozen registry from `tools({...})` | +| `[clientToolContinuationPolicy]` | Cap runaway continuation loops (default 10 groups per turn) | +| `[clientToolExecutionGuard]` | Durable claim-before-execute for non-idempotent tools | + + + The `settle` / `flush` / `resolve` contract behind these features is documented in [Writing an Adapter › Client Tools](/docs/chat/guides/writing-an-adapter#client-tools-optional). + ## What's next diff --git a/apps/website/content/docs/chat/guides/writing-an-adapter.mdx b/apps/website/content/docs/chat/guides/writing-an-adapter.mdx index 30db34033..faec1667d 100644 --- a/apps/website/content/docs/chat/guides/writing-an-adapter.mdx +++ b/apps/website/content/docs/chat/guides/writing-an-adapter.mdx @@ -264,6 +264,46 @@ How adapters in this repo answer the question: If your protocol does store thread state, follow the LangGraph adapter's pattern: react to `threadId` changes, fetch the latest checkpoint, and surface a `isThreadLoading` signal so the UI can show a skeleton while the fetch runs. If it doesn't, follow AG-UI and be explicit in your docs that consumers own the load step. +## Client Tools (Optional) + +If your runtime lets the browser declare tools the model can call, implement the optional `clientTools` capability. Consumers then get [client tools](/docs/chat/guides/client-tools) with no further work. + +```typescript +import type { ClientToolsCapability } from '@threadplane/chat'; +``` + +| Member | Required | What you supply | +|---|---|---| +| `setCatalog(specs)` | yes | Store the catalog; ship it with every outbound run | +| `pending` | yes | `Signal` — calls awaiting a browser result | +| `resolve(id, result)` | yes | Record the result **and continue the run** | +| `settle?(id, result)` | no | Record the result **without** continuing | +| `flush?()` | no | Make everything recorded via `settle` durable, still without continuing | + +Use `selectPendingClientToolCalls` from `@threadplane/chat` for `pending` rather than hand-rolling the predicate — a call is pending when the run is **not** in flight, its name is in the catalog, it has no backend result, and it has not already been settled locally. + +### The invariant + +> **The server thread must never hold a client tool call without a corresponding tool result.** + +Violate it and the thread carries an assistant message with `tool_calls` and no matching tool message; most providers reject that history outright on the next user turn. + +`resolve()` alone cannot uphold this, because some groups never continue — every tool in them is terminal (`followUp: false`), the user pressed stop, or a continuation limit tripped. That is what `settle` + `flush` are for. + + + If your `settle()` only records locally, you **must** implement `flush()` to write those results to the server. `@threadplane/ag-ui` gets this for free — its `settle()` calls `addMessage()`, which places the message in the outgoing list, so `flush()` is a no-op. `@threadplane/langgraph` buffers instead, so its `flush()` writes the whole batch in one `threads.updateState` call. Omitting `flush()` when your `settle()` is not already durable silently drops results. + + +### Implementation rules + +Three rules, each learned from a real defect: + +1. **Take ownership of the batch when the write starts, not when it finishes.** Remove the buffered entries up front and re-stage them only if the write fails. Leaving them in place across the `await` lets a concurrent `resolve()` re-send what you are already writing (two tool messages for one call) and lets your completion handler remove the wrong entries. +2. **Coalesce concurrent flushes by chaining, not short-circuiting.** Returning an in-flight promise to a second caller resolves *their* `flush()` without ever writing *their* batch. +3. **Discard staged results on a thread switch.** A tool message only makes sense against the thread whose assistant message produced its `tool_call_id`. Note that a thread id captured at settle time may already be the *new* thread if your `setThreadId` is synchronous — key the check on something that survives the switch. + +Adapters without a durable write path can fall back to attaching staged results to the next outbound run. That is correct in the normal path but loses them across a reload, so document the limitation. + ## Publishing Your Adapter Want to distribute your adapter as an npm package? Keep the following in mind.