From 52f2bbe3e268d47227fa5f055a062f35688b773b Mon Sep 17 00:00:00 2001 From: Brian Love Date: Sun, 9 Aug 2026 06:24:36 -0700 Subject: [PATCH 1/2] docs: design missing persistence flush error --- ...ggraph-flush-missing-persistence-design.md | 118 ++++++++++++++++++ 1 file changed, 118 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-09-langgraph-flush-missing-persistence-design.md diff --git a/docs/superpowers/specs/2026-08-09-langgraph-flush-missing-persistence-design.md b/docs/superpowers/specs/2026-08-09-langgraph-flush-missing-persistence-design.md new file mode 100644 index 000000000..ab6a7ade4 --- /dev/null +++ b/docs/superpowers/specs/2026-08-09-langgraph-flush-missing-persistence-design.md @@ -0,0 +1,118 @@ +# LangGraph Flush Missing-Persistence Error + +**Date:** 2026-08-09 +**Status:** Approved for implementation +**Builds on:** `2026-08-07-client-tool-continuation-fixes-design.md` + +## Problem + +LangGraph client-tool results settled without a follow-up run are buffered in +the browser until `flush()` makes them durable with `AgentTransport.updateState`. +Custom transports may omit `updateState`, so `agent.fn.ts` constructs the +client-tools capability without a persistence function. + +Today `flush()` returns a resolved promise when that persistence function is +missing, even when the buffer contains results. The buffer survives for the +next ordinary `submit()`, but terminal groups do not require another submit. +A reload before the next user message therefore loses the results while every +caller observed a successful flush. + +This violates the method's public contract: a resolved `flush()` must mean the +staged results are durable. + +## Decision + +`flush()` will reject when both conditions are true: + +1. one or more client-tool results are staged; and +2. no persistence function is configured. + +The error message will explain that custom LangGraph transports using terminal +client tools must implement `updateState()`. + +An empty-buffer `flush()` remains a successful no-op. This preserves harmless +calls made as part of generic settlement flows when there is nothing to write. + +## Data and Control Flow + +The check belongs in `createClientToolsCapability`, where both the staged +buffer and the optional persistence function are visible. + +1. `settle()` records the local result and appends its `ToolMessage` to the + staged buffer. +2. `flush()` checks whether the buffer is empty. If so, it resolves. +3. If the buffer is non-empty and persistence is unavailable, `flush()` rejects + without taking ownership of or mutating the buffer. +4. If persistence is available, the existing batching, chaining, thread + filtering, and re-staging logic runs unchanged. +5. A later ordinary `submit()` may still drain the retained buffer. This + fallback limits damage, but it no longer masquerades as successful + durability. + +The chat coordinator already catches rejected flush promises and logs +`Client tool flush failed`, so terminal settlement will not create an +unhandled rejection or start a continuation run. + +## Error Handling + +- **Empty buffer:** resolve without requiring persistence. +- **Missing persistence with staged results:** reject with a configuration + error and retain every staged result. +- **Configured persistence rejects:** retain the existing behavior: warn, + re-stage the owned batch, and allow a later flush or submit to retry it. +- **Thread switch:** retain the existing generation and retired-tool-call + guards; this change must not re-stage results onto another thread. + +The configured-persistence failure behavior is intentionally out of scope. It +has different retry and observability trade-offs from the deterministic +missing-capability error. + +## Alternatives Rejected + +### Reject at agent construction + +Failing whenever a custom transport omits `updateState()` would reject valid +agents that never use terminal client tools. The error should occur only when +durability is actually requested for staged data. + +### Start a continuation run automatically + +Submitting the buffered results would contradict `followUp: false`, user-stop, +and continuation-limit semantics. `flush()` must never start a run. + +### Warn and resolve + +A warning preserves the false success signal. Callers awaiting `flush()` need +an observable failure to distinguish durable state from volatile fallback +state. + +## Testing + +### Capability tests + +- A staged result plus no persistence function rejects with the configuration + error. +- The rejected flush leaves the complete buffer available to + `drainToolMessages()`. +- No persistence function plus an empty buffer still resolves. +- Existing successful, failed, concurrent, and thread-switch flush tests remain + green. + +### Agent wiring test + +An agent using a custom transport without `updateState()` rejects `flush()` +after settlement and still drains that result into the next ordinary submit. +This verifies the public adapter seam rather than only the internal factory. + +### Documentation + +Update the client-tools guide to state that terminal settlement rejects when a +custom LangGraph transport cannot persist results, while the next-submit +fallback remains available if the application chooses to recover. + +## Out of Scope + +- Changing the `ClientToolsCapability` type or adding a public error class. +- Changing AG-UI, whose settlement path is already durable. +- Changing how configured persistence failures are logged or propagated. +- Fixing the separate destructive-drain behavior when `submit()` fails. From e98a050119966974e0310ae8c6ab2e459af94378 Mon Sep 17 00:00:00 2001 From: Brian Love Date: Sun, 9 Aug 2026 06:43:37 -0700 Subject: [PATCH 2/2] fix(langgraph): reject non-durable client tool flushes --- .../content/docs/chat/guides/client-tools.mdx | 2 +- ...-09-langgraph-flush-missing-persistence.md | 237 ++++++++++++++++++ libs/langgraph/src/lib/agent.fn.spec.ts | 4 +- libs/langgraph/src/lib/agent.fn.ts | 4 +- libs/langgraph/src/lib/client-tools.spec.ts | 19 +- libs/langgraph/src/lib/client-tools.ts | 18 +- 6 files changed, 272 insertions(+), 12 deletions(-) create mode 100644 docs/superpowers/plans/2026-08-09-langgraph-flush-missing-persistence.md diff --git a/apps/website/content/docs/chat/guides/client-tools.mdx b/apps/website/content/docs/chat/guides/client-tools.mdx index 8b643a53d..819b96b10 100644 --- a/apps/website/content/docs/chat/guides/client-tools.mdx +++ b/apps/website/content/docs/chat/guides/client-tools.mdx @@ -126,7 +126,7 @@ const clientTools = tools({ Follow-up is decided **per tool-call group**, not per tool. If the model calls three tools in one turn and any one of them wants a follow-up, the whole group continues in a single run once every result has settled. Only when *every* tool in the group is terminal does the turn end. - A terminal group has no follow-up run to carry its results, so the adapter writes them to the server directly. On `@threadplane/langgraph` that uses the transport's `updateState`. A **custom transport that does not implement `updateState`** falls back to attaching the results to the user's *next* message — correct in the normal path, but the results are lost if the page reloads first, leaving a tool call with no result and a provider error on the next turn. If you supply your own transport and use terminal tools, implement `updateState`. + A terminal group has no follow-up run to carry its results, so the adapter writes them to the server directly. On `@threadplane/langgraph` that uses the transport's `updateState`. If a **custom transport does not implement `updateState`**, `flush()` rejects when terminal results are staged. The results stay buffered and an ordinary next message can still carry them, but a reload first loses them and leaves the server thread with an unanswered tool call. If you supply your own transport and use terminal tools, implement `updateState`. ## Re-running tools safely with `idempotent` diff --git a/docs/superpowers/plans/2026-08-09-langgraph-flush-missing-persistence.md b/docs/superpowers/plans/2026-08-09-langgraph-flush-missing-persistence.md new file mode 100644 index 000000000..ae745e0f5 --- /dev/null +++ b/docs/superpowers/plans/2026-08-09-langgraph-flush-missing-persistence.md @@ -0,0 +1,237 @@ +# LangGraph Flush Missing-Persistence Error Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make LangGraph `flush()` reject when staged client-tool results cannot be persisted, while preserving empty-buffer no-op behavior and the next-submit fallback. + +**Architecture:** Keep the behavior inside `createClientToolsCapability`, the only unit that sees both the staged buffer and optional persistence function. Add capability and agent-seam regression coverage before changing production code, then update the user-facing transport warning. Preserve every existing batching, failure-restaging, concurrency, and thread-switch path. + +**Tech Stack:** TypeScript, Angular signals, Vitest, Nx, MDX. + +**Design:** `docs/superpowers/specs/2026-08-09-langgraph-flush-missing-persistence-design.md` + +--- + +## File Map + +- Modify `libs/langgraph/src/lib/client-tools.spec.ts`: define the internal capability contract for empty and non-empty flushes without persistence. +- Modify `libs/langgraph/src/lib/agent.fn.spec.ts`: verify the public LangGraph agent seam rejects and still drains on the next ordinary submit. +- Modify `libs/langgraph/src/lib/client-tools.ts`: implement the missing-persistence rejection without taking ownership of the buffer. +- Modify `libs/langgraph/src/lib/agent.fn.ts`: keep the transport-wiring comment aligned with the observable rejection and retained fallback. +- Modify `apps/website/content/docs/chat/guides/client-tools.mdx`: document the observable rejection and retained fallback. + +No API or narrative-doc generator is required: no public type/JSDoc surface changes, and the edited guide is source MDX rather than generated content. `generate-agent-context` is also unnecessary because it reads `CLAUDE.md.template` and `AGENTS.md.template`, neither of which changes. + +### Task 1: Reproduce the false-success contract at both LangGraph seams + +**Files:** +- Modify: `libs/langgraph/src/lib/client-tools.spec.ts:433-457` +- Modify: `libs/langgraph/src/lib/agent.fn.spec.ts:1377-1396` + +- [ ] **Step 1: Add the empty-buffer capability test** + +Extend the existing empty-buffer case so it covers an omitted persistence function as well as a configured one: + +```ts +it('resolves without persistence when the buffer is empty', async () => { + const { cap } = setup(undefined); + + await expect(cap.flush?.()).resolves.toBeUndefined(); +}); +``` + +Keep the existing configured-persistence empty-buffer test because it separately proves that no server call is made. + +- [ ] **Step 2: Replace the silent-success capability test with the rejected contract** + +Replace `keeps the buffer when no persist function is supplied` with: + +```ts +it('rejects and keeps the buffer when staged results have no persistence function', async () => { + const { cap } = setup(undefined); + cap.settle?.('t1', { ok: true, value: 'a' }); + cap.settle?.('t2', { ok: true, value: 'b' }); + + await expect(cap.flush?.()).rejects.toThrow( + 'Custom LangGraph transports using terminal client tools must implement updateState().', + ); + expect(cap.drainToolMessages()).toEqual([ + { type: 'tool', role: 'tool', tool_call_id: 't1', content: 'a' }, + { type: 'tool', role: 'tool', tool_call_id: 't2', content: 'b' }, + ]); +}); +``` + +- [ ] **Step 3: Make the agent-wiring fallback test expect rejection before recovery** + +In `submit drains staged tool messages ahead of the payload messages`, change the flush assertion to: + +```ts +cap.settle('tc-1', { ok: true, value: 'sunny' }); +await expect(cap.flush()).rejects.toThrow( + 'Custom LangGraph transports using terminal client tools must implement updateState().', +); +ref.submit({ message: 'and tomorrow?' }); +``` + +Keep the payload-order and exact-once assertions. They prove the rejection does not destroy the fallback buffer. + +- [ ] **Step 4: Run the focused tests and verify RED** + +Run: + +```bash +npx nx test langgraph --skip-nx-cache -- -t "flush|submit drains staged" +``` + +Expected: FAIL because both staged-result cases resolve instead of rejecting. The new empty-buffer case must pass, proving the failure is specific to buffered data. + +### Task 2: Implement the minimal capability-level rejection + +**Files:** +- Modify: `libs/langgraph/src/lib/client-tools.ts:266-299` +- Modify: `libs/langgraph/src/lib/agent.fn.ts:436-442` + +- [ ] **Step 1: Move the missing-persistence decision behind the empty-buffer check** + +Change the start of `runFlush()` to inspect the buffer before deciding whether persistence is required: + +```ts +function runFlush(): Promise { + if (toolMessageBuffer.length === 0) return Promise.resolve(); + if (!persistFn) { + return Promise.reject( + new Error( + 'Cannot flush staged client tool results. ' + + 'Custom LangGraph transports using terminal client tools must implement updateState().', + ), + ); + } + + const staged = takeStagedForCurrentThread(); + if (staged.length === 0) return Promise.resolve(); +``` + +Remove the early `if (!persistFn) return Promise.resolve();` from `flush()` so every non-concurrent call reaches `runFlush()`: + +```ts +flush(): Promise { + if (flushInFlight) { + // existing chaining logic unchanged + } + return runFlush(); +}, +``` + +The rejection occurs before `takeStagedForCurrentThread()`, so the buffer is unchanged and remains available to the next-submit fallback. + +- [ ] **Step 2: Update the factory contract comment** + +Replace the claim that an absent `persistFn` merely degrades silently. State that a non-empty flush rejects and retains the buffer for an explicit recovery through a later submit. + +- [ ] **Step 3: Update the agent transport-wiring comment** + +Keep `agent.fn.ts` aligned with the capability contract: + +```ts +// When persistFn is undefined, a non-empty flush() rejects and keeps the +// buffer staged; the submit wrapper below can still drain it into the next run. +``` + +- [ ] **Step 4: Run the focused tests and verify GREEN** + +Run: + +```bash +npx nx test langgraph --skip-nx-cache -- -t "flush|submit drains staged" +``` + +Expected: PASS. Confirm the missing-persistence cases reject, the empty-buffer case resolves, and existing flush concurrency/failure tests remain green. + +### Task 3: Document the runtime contract + +**Files:** +- Modify: `apps/website/content/docs/chat/guides/client-tools.mdx:128-130` + +- [ ] **Step 1: Update the custom-transport warning** + +Revise the warning to say: + +```mdx + + A terminal group has no follow-up run to carry its results, so the adapter writes them to the server directly. On `@threadplane/langgraph` that uses the transport's `updateState`. If a **custom transport does not implement `updateState`**, `flush()` rejects when terminal results are staged. The results stay buffered and an ordinary next message can still carry them, but a reload first loses them and leaves the server thread with an unanswered tool call. If you supply your own transport and use terminal tools, implement `updateState`. + +``` + +- [ ] **Step 2: Verify the guide compiles** + +Run: + +```bash +npx nx build website --skip-nx-cache +``` + +Expected: PASS with the updated MDX compiled into the site. + +### Task 4: Full scoped verification and one logical commit + +**Files:** +- Review all modified files from Tasks 1-3. + +- [ ] **Step 1: Run LangGraph tests** + +Run: + +```bash +npx nx test langgraph --skip-nx-cache +``` + +Expected: PASS. + +- [ ] **Step 2: Run LangGraph lint** + +Run: + +```bash +npx nx lint langgraph --skip-nx-cache +``` + +Expected: PASS with zero errors. + +- [ ] **Step 3: Build the LangGraph package** + +Run: + +```bash +npx nx build langgraph --skip-nx-cache +``` + +Expected: PASS. + +- [ ] **Step 4: Review repository state and diff** + +Run: + +```bash +git status --short +git diff --check +git diff --stat +git diff +``` + +Expected: only the two LangGraph test files, `client-tools.ts`, the aligned comment in `agent.fn.ts`, the client-tools guide, and this plan are changed after the already-committed design spec; no whitespace errors or generated dependency changes. + +- [ ] **Step 5: Commit the completed implementation once** + +The repository contributor guide forbids mid-task commits, so group the finished code, tests, docs, and plan in one logical commit: + +```bash +git add \ + libs/langgraph/src/lib/client-tools.ts \ + libs/langgraph/src/lib/client-tools.spec.ts \ + libs/langgraph/src/lib/agent.fn.ts \ + libs/langgraph/src/lib/agent.fn.spec.ts \ + apps/website/content/docs/chat/guides/client-tools.mdx \ + docs/superpowers/plans/2026-08-09-langgraph-flush-missing-persistence.md +git commit -m "fix(langgraph): reject non-durable client tool flushes" +``` diff --git a/libs/langgraph/src/lib/agent.fn.spec.ts b/libs/langgraph/src/lib/agent.fn.spec.ts index 24ca4a279..9a576ac6b 100644 --- a/libs/langgraph/src/lib/agent.fn.spec.ts +++ b/libs/langgraph/src/lib/agent.fn.spec.ts @@ -1385,7 +1385,9 @@ describe('agent — client tool staging', () => { cap.setCatalog([SPEC]); cap.settle('tc-1', { ok: true, value: 'sunny' }); - await cap.flush(); + await expect(cap.flush()).rejects.toThrow( + 'Custom LangGraph transports using terminal client tools must implement updateState().', + ); ref.submit({ message: 'and tomorrow?' }); const payload = transport.streams[0]?.payload as { messages: Array> }; diff --git a/libs/langgraph/src/lib/agent.fn.ts b/libs/langgraph/src/lib/agent.fn.ts index 2bea4ffdd..8f623059c 100644 --- a/libs/langgraph/src/lib/agent.fn.ts +++ b/libs/langgraph/src/lib/agent.fn.ts @@ -437,8 +437,8 @@ export function agent< // updateState() silently no-ops when the transport has no updateState, so // only supply a persist function when the effective transport supports it — // an omitted transport means the bridge builds a FetchStreamTransport, which - // does. When persistFn is undefined, flush() keeps the buffer and the submit - // wrapper below drains it into the next run instead. + // does. When persistFn is undefined, a non-empty flush() rejects and keeps the + // buffer staged; the submit wrapper below can still drain it into the next run. const canPersistToolMessages = !transport || typeof transport.updateState === 'function'; const clientToolsCap = createClientToolsCapability( (payload, opts) => manager.submit(payload, opts), diff --git a/libs/langgraph/src/lib/client-tools.spec.ts b/libs/langgraph/src/lib/client-tools.spec.ts index c13e7dad2..dbc9d6d83 100644 --- a/libs/langgraph/src/lib/client-tools.spec.ts +++ b/libs/langgraph/src/lib/client-tools.spec.ts @@ -437,6 +437,12 @@ describe('flush', () => { expect(persist).not.toHaveBeenCalled(); }); + it('resolves without persistence when the buffer is empty', async () => { + const { cap } = setup(undefined); + + await expect(cap.flush?.()).resolves.toBeUndefined(); + }); + it('keeps the buffer when persist fails so a later drain retries', async () => { const persist = vi.fn(async () => { throw new Error('boom'); }); const { cap } = setup(persist); @@ -449,11 +455,18 @@ describe('flush', () => { ]); }); - it('keeps the buffer when no persist function is supplied', async () => { + it('rejects and keeps the buffer when staged results have no persistence function', async () => { const { cap } = setup(undefined); cap.settle?.('t1', { ok: true, value: 'a' }); - await cap.flush?.(); - expect(cap.drainToolMessages()).toHaveLength(1); + cap.settle?.('t2', { ok: true, value: 'b' }); + + await expect(cap.flush?.()).rejects.toThrow( + 'Custom LangGraph transports using terminal client tools must implement updateState().', + ); + expect(cap.drainToolMessages()).toEqual([ + { type: 'tool', role: 'tool', tool_call_id: 't1', content: 'a' }, + { type: 'tool', role: 'tool', tool_call_id: 't2', content: 'b' }, + ]); }); it('empties the buffer after a successful flush', async () => { diff --git a/libs/langgraph/src/lib/client-tools.ts b/libs/langgraph/src/lib/client-tools.ts index 0bacdacf8..0fa318417 100644 --- a/libs/langgraph/src/lib/client-tools.ts +++ b/libs/langgraph/src/lib/client-tools.ts @@ -134,9 +134,10 @@ export function mergeStagedToolMessages( * - flush(): makes the whole buffer durable in ONE persistFn call without * starting a run — the settlement path for tool groups that never continue. * The batch leaves the buffer at snapshot time and is re-staged only if the - * write fails, so a failure (or an absent persistFn) still degrades to the - * next flush or to the drainToolMessages() fallback in the submit wrapper, - * while a concurrent resolve()/drain can never re-send an in-flight batch. + * write fails. When persistFn is absent, a non-empty flush rejects without + * taking ownership of the buffer, retaining it for explicit recovery through + * the later-submit drainToolMessages() fallback. A concurrent resolve()/drain + * can never re-send an in-flight batch. * - clearStagedToolMessages(): discards the buffer on a thread switch. * - resolve(id, result): settles the result, then issues a NEW run on the SAME * thread by calling submitFn with the full buffered ToolMessage group: @@ -271,7 +272,15 @@ export function createClientToolsCapability( * by the wrong index (dropping a result that was never persisted). */ function runFlush(): Promise { - if (!persistFn) return Promise.resolve(); + if (toolMessageBuffer.length === 0) return Promise.resolve(); + if (!persistFn) { + return Promise.reject( + new Error( + 'Cannot flush staged client tool results. ' + + 'Custom LangGraph transports using terminal client tools must implement updateState().', + ), + ); + } const staged = takeStagedForCurrentThread(); if (staged.length === 0) return Promise.resolve(); @@ -337,7 +346,6 @@ export function createClientToolsCapability( }, flush(): Promise { - if (!persistFn) return Promise.resolve(); if (flushInFlight) { // Chain rather than short-circuit. The caller's batch may have been // staged AFTER the in-flight write took its snapshot, so returning that