Repository navigation
fix: detect reasoning loops from streamed text in a worker - #1878
Merged
Merged
Conversation
Replace the between-calls loop-detection middleware, which could never see repetition inside a single generation, with a streaming detector. Reasoning and message deltas are split into sentence segments and hashed into a rolling window; when a segment repeats past the threshold the stream is abandoned and the turn re-run with a looping nudge, compacting context after streamMaxNudges failed nudges.
Build the detector on coalesceSegments so a segment is the same message/reasoning block the TUI renders (docs/STREAMING.md), instead of a custom sentence split. Drop maxBuffer, minLength and flush; short segments such as 'ok' are now counted.
Stream the repeating paragraph from madz_loop.md as multi-character reasoning_content deltas (1-20 chars, cut mid-word, no ids) through consumeStream with the segment detector. Covers one paragraph, two paragraphs, two sentences and one sentence repeating, with paragraph, line and space separators. These tests currently fail: the detector never closes a segment on an id-less delta stream.
A segment is one streamed delta (a small group of characters), not a sentence or block. Hash each delta into a bounded per-kind list and flag a loop when the tail is one unit repeated back-to-back more than threshold times, at any period. Default windowSize is now 1000 so three copies of a paragraph fit; ttlMs is removed. Replay lines 14-21 of the captured loop as random 4-10 char segments repeated, plus 1-20 char delta variants across paragraph, sentence and separator shapes.
A segment is one streamed delta (a small group of characters), not a sentence or block. Hash each delta into a bounded per-kind list and flag a loop when the tail is one unit repeated back-to-back more than threshold times, at any period. Default windowSize is now 1000 so three copies of a paragraph fit; ttlMs is removed. Replay lines 14-21 of the captured loop as random 4-10 char segments repeated through Chat Completions reasoning_content, Responses API reasoning blocks and message text, asserting identical detection.
Stop retrying inside runAgent. It now returns loopDetected from a single stream, and the conversation area dispatches a silent "You're looping" via handleChat with silentUser, mirroring the reasoning-only Continue. shouldNudgeLoop gates it on the last bubble and a nudge count bounded by loopDetection.streamMaxNudges, reset per session.
Compute loopNudgeLimit once per component instead of per turn, and drop the hardcoded ?? 2 fallback; the schema already defaults streamMaxNudges.
Compute autoContinueLimit once per component instead of per turn, and drop the hardcoded ?? 1000 fallback; the agent schema already defaults it.
Put escalationCount, escalationWindowMs and ttlMs back in the schema and config.yaml, and wire them up. The detector prunes segment hashes older than ttlMs. The TUI tracks nudge timestamps and compacts the context once escalationCount nudges fall within escalationWindowMs, replacing streamMaxNudges.
Dispatch "You're looping" without silentUser so it renders as a user message in the TUI and it is obvious when the loop detector fires.
Hashing each streamed delta missed loops whenever the repeats were cut at different positions, which is how a provider really streams: only single-character deltas were detected. Accumulate the streamed text per kind and check the text instead: a sentence repeated within the last windowSize completed sentences, or a run of words repeated back-to-back for loops with no punctuation such as 'ok ok ok'. The check runs in a Piscina worker so it stays off the main loop. At most one check per kind is in flight, text that arrives meanwhile is re-checked, a failing worker is treated as 'not looping', and the pool is destroyed on shutdown. windowSize now means recent sentences (default 20). Tests cut every repeat at its own random positions.
The detector is confirmed firing, so dispatch "You're looping" with silentUser like the reasoning-only Continue.
The detector re-split the whole accumulated text for each kind on every check and copied it to the worker, so cost grew with stream length: about 7 ms per check at 3 MB, quadratic over a long run. Split only the new text as it arrives and keep the last windowSize sentences and wordWindowSize words. The worker now receives those bounded lists, and a check only starts when a delta completed a sentence or word. Per-push cost is flat (about 1 us from 1k to 200k deltas). Verdicts are unchanged however the stream is chunked. Adds loopLoop wordWindowSize handling in the detector only; no config change.
Add loopDetection.wordWindowSize (default 1000) to the schema, config.yaml and index.js. The back-to-back word check has to hold threshold + 1 copies of the longest repeating unit: the two-paragraph capture is 190 words, so three copies need 570 and the previous hardcoded 200 could not see it. Measured on a no-match worst case, the check stays at a few microseconds and the task copy is about 19 us at 1000 words. Also correct the schema header, which still described hashing deltas.
State was keyed by kind alone, so concurrent blocks interleaved into one sentence and word list. That could hide a real loop in one block, or invent one from two blocks that merely alternate. This regressed an earlier version that passed the block id through to the detector. Key state by kind plus the block id: the content-block index for message text and block.id for Responses API reasoning. Chat Completions reasoning_content carries no id and stays keyed by kind. Each block gets its own lists and its own in-flight check. At most maxBlocks (16) blocks are tracked and the least recently updated is evicted, so memory stays bounded. Stale-answer handling moves from a global generation counter to a per-block dead flag, so expiring or evicting one block no longer discards another block's in-flight result.
A stream is a kind plus an id, not a block, and a long turn can hold hundreds to thousands of them. Rename the option to maxStreams and set the default to 1000, backed by a measurement: a stream with both windows full holds about 30-40 KB, so 1000 is roughly 30-40 MB worst case. Add maxStreams to the loopDetection schema, config.yaml and index.js, and document that a looping stream is always receiving deltas so eviction, which drops the least recently updated stream, never removes it. Tests and wording updated from block to stream.
consumeStream now resets the loop detector in a finally block, so its word and sentence lists are cleared and nulled when the turn ends however it ends: finished, interrupted by a clarify, abandoned on a detected loop, or thrown. A released state is also marked dead, so a check still running in the worker cannot flag a stream that has been released. maxStreams default goes from 1000 to 4: two concurrent streams is the realistic edge case and four leaves headroom, since finished streams are released and a long turn of sequential messages never accumulates them. Adds size() to the detector so tests can assert on what is tracked.
A message turn is start, stream, end, so the detector's word and sentence lists are cleared and nulled when each message ends, not only when the whole agent run does. Without this, a run with hundreds of messages kept state for the last few until they were evicted. consumeStream now ends a message when its last chunk carries a finish_reason, when a different message id arrives, or when a tool result follows the AI message. end() reads the verdict of any check still running and then releases, so a loop in the final deltas is not lost. Subagent events are ignored for this. The stream-end reset stays as a backstop. Also stops the same short answer repeated across separate messages from being counted as one loop.
Three strikes and you're out. Each detected loop is a strike; strikes within escalationWindowMs are counted including the current one, and the escalationCount-th strike compacts the context instead of nudging. So strikes 1 and 2 nudge and strike 3 compacts. The previous code waited for three nudges to be on record, so it compacted on the fourth loop. Update the decideLoopAction tests to play out strikes 1 to 3 and to cover window pruning, the reset after compaction and other escalationCount values, and correct the schema and component comments.
When the third loop strike compacts the context, resume the model with a silent "Continue" so it does not sit idle until the user types. handleCompact now returns whether it succeeded, and shouldContinueAfterCompact gates the Continue on that and on the shared autoContinueLimit, so a failed compaction (the looping context is still in place) does not continue and a model that keeps looping cannot cycle through nudge, compact and continue forever. The /compact command ignores the return value.
Resolve the conflict in src/tui/conversationArea.js between this branch's loop strike handling and main's #1879, which widened shouldAutoContinue to turns that end on a tool message and skips interrupted turns. Keep both: the loop nudge/compact block stays ahead of auto-continue, and the auto-continue comment takes main's wording. Apply main's interrupted rule to the loop path as well. A turn paused on a clarify interrupt is waiting for the user's answer, so decideLoopAction now ignores it and records no strike, rather than nudging or compacting a paused turn. autoContinueLimit stays read once from config, so main's ?? 1000 fallback in runTurn is not reintroduced.
6 of 12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Reasoning loops were not being detected: the
LoopDetectionmiddleware hashed whole messages inbeforeModel, so repetition inside a single streamed generation never reached it. This removes the middleware and adds a streaming detector.Detection. Each streamed delta is split once, as it arrives, into completed sentences and words, and only the last
windowSizesentences andwordWindowSizewords are kept, so memory and per-delta cost stay bounded however long a stream runs and the provider's delta boundaries do not matter. A loop is a sentence repeated more thanthresholdtimes within the sentence window, or a run of words repeated back-to-back (for loops with no punctuation, such asok ok ok). State is kept per stream, keyed by kind plus the stream's id (the content-block index for message text,block.idfor Responses API reasoning; Chat Completionsreasoning_contenthas no id and is keyed by kind), so concurrent streams never mix. At mostmaxStreamsare tracked, least recently updated evicted. The check runs in a Piscina worker thread, off the main loop. When a message turn ends (afinish_reason, a different message id, or a tool result), its lists are cleared and nulled.Response. A flagged stream is abandoned. Each detected loop is a strike within
escalationWindowMs: strikes 1 and 2 send a silent "You're looping" throughhandleChatwithsilentUser: true, and strikeescalationCount(3) compacts the context instead, then sends a silent "Continue" if the compaction succeeded (counted againstautoContinueLimit). A turn paused on a clarify interrupt is never nudged or compacted, matching #1879's rule for auto-continue.Config (
loopDetectioninconfig.yaml), changed frommain:windowSize10 to 20 (it now counts sentences), and newwordWindowSize(1000) andmaxStreams(4).enabled,threshold,ttlMs,escalationCountandescalationWindowMsare unchanged.Type of Change
Testing
tests/unit/stream/loopCheck.test.js: the pure text checks, including that sentences and words come out identical however the text is chunked.tests/unit/stream/loopDetector.test.js: the detector (okwith no punctuation, a repeated sentence, no false positive on distinct text, one check in flight, re-check of text that arrives mid-check, a failing worker treated as "not looping",ttlMs, bounded window and task size, a real worker thread) andconsumeStreamaborting and closing the stream.tests/unit/stream/loopDetector.sample.test.js: replays the captured loop (tests/fixtures/loop-reasoning.md) with every repeat cut at its own random positions, through Chat Completionsreasoning_content, Responses API reasoning blocks and message text; includesokx20 and one sentence x5, the two cases that failed in practice.tests/unit/stream/loopDetectorStreams.test.js: per-stream keying (interleaved streams, id 0, eviction,ttlMsper stream).tests/unit/stream/loopDetectorTurnEnd.test.jsandloopDetectorMessageEnd.test.js: release on stream end, interrupt, abandon and throw, and at each message end (finish_reason, new id, tool result); 400 sequential messages never accumulate.tests/unit/stream/loopDetectorWindow.test.js: schema defaults and bounds forwordWindowSizeandmaxStreams, and that 1000 words holds three copies of the two-paragraph capture.tests/unit/tui/conversationArea.test.js:decideLoopAction(strikes 1 to 3, window pruning, reset after compaction, no action on an interrupted turn) andshouldContinueAfterCompact.Not covered: the
handleCompactcall and the "Continue" dispatch inrunTurnhave no test (only the decision functions do, andconversationArea.jsis excluded from coverage). Detection lags one worker round trip behind the stream. Nothing has been run against a live model stream, so real provider chunking, and whether providers setfinish_reasonand a stable message id on streamed chunks, is unverified. One full-suite run showed a singleloopDetector.test.jsfailure that did not reproduce in 9 reruns; its cause is unknown.Coverage
loopCheck.js,loopDetector.jsandloopDetectorWorker.jsare at 100% lines;consume.jsrose from 85.5% to 96.5% (fromcoverage.txt).Checklist
npm run lintpassesRemoves
src/provider/loopDetectionMiddleware.js, its test file and its wiring insrc/agent/deepAgents.js.