Skip to content

Fix chat SSE timeout on long-running Temporal-engine turns - #209

Merged
imaustink merged 1 commit into
mainfrom
fix/temporal-engine-stream-blocks-heartbeat
Aug 17, 2026
Merged

Fix chat SSE timeout on long-running Temporal-engine turns#209
imaustink merged 1 commit into
mainfrom
fix/temporal-engine-stream-blocks-heartbeat

Conversation

@imaustink

Copy link
Copy Markdown
Owner

Summary

Root-causes the chat-UI timeout ("Timeout or cancellation waiting for update: stream terminated by RST_STREAM with error code: CANCEL") seen on a long-running agent turn, even though the underlying agentrun-* Kubernetes Job kept running and completed successfully server-side.

TemporalEngine.stream() (apps/agent-orchestrator/src/engine/temporal-engine.ts) did await this.invoke(input) before returning its iterable. server.ts's streaming handler does:

const source = await this.graph.stream(graphInput, { streamMode: "updates" });
for await (const item of withHeartbeat(source, HEARTBEAT_MS)) { ... }

withHeartbeat exists specifically to emit an SSE keep-alive comment whenever the source stalls for longer than HEARTBEAT_MS (15s) — but it can't race against a source it doesn't have yet. Because stream()'s own Promise didn't settle until the entire turn (up to the 30-minute poll timeout) had already finished, the heartbeat wrapper never engaged for the whole duration of a Temporal-engine turn. The only bytes reaching the client during that window were opportunistic progress-narration writes (added in #208) — not a guaranteed cadence — so any lull (job scheduling/image pull, a long tool call, or simply the seconds before the first narration line) longer than an upstream idle-connection timeout got the stream cancelled.

Fix

Move the await this.invoke(input) inside the returned async generator's body instead of before the return. Async generator bodies don't run until the first .next() call, so stream() now resolves immediately, letting withHeartbeat start racing the real turn duration against its keep-alive timer from the start — exactly like it already does for the LangGraph engine.

No behavior change for a non-streaming caller (invoke() is untouched) or for what gets rendered — still a single terminal update once the turn settles.

Test plan

  • Added a regression test (temporal-engine.test.ts) asserting stream() resolves promptly even while the underlying turn is still "hanging", and that iterating the returned source still drains through to the terminal result once the turn completes.
  • npx vitest run src/engine/temporal-engine.test.ts — 12/12 passing.
  • Full apps/agent-orchestrator suite run for regressions — the 2 failing files (nats-agent-channel*.test.ts) and the pre-existing tsc --noEmit errors in nats-agent-channel.ts reproduce identically on main without this change (unrelated, stale workspace-package typing issue).

stream() used to `await this.invoke(input)` before returning the
iterable, so its own Promise didn't settle until the whole turn (up to
30 min) had already finished. server.ts's streaming handler does
`const source = await this.graph.stream(...)` before starting
`withHeartbeat(source, HEARTBEAT_MS)`, so that wrapper never got a
source to race against until the turn was over -- a long-running turn's
SSE connection had no guaranteed keep-alive byte cadence at all, and an
idle-connection timeout upstream (ingress/ALB/browser) would cancel it
(RST_STREAM) even though the AgentRun Job kept running and completed
fine server-side.

Moving the `await this.invoke(input)` inside the async generator body
defers it until withHeartbeat's first `.next()` call, which is exactly
when its race against HEARTBEAT_MS needs to start.
@imaustink
imaustink merged commit cdcb8dd into main Aug 17, 2026
6 checks passed
@imaustink
imaustink deleted the fix/temporal-engine-stream-blocks-heartbeat branch August 17, 2026 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant