Repository navigation
fix(agent): trigger checkpoints on the full context size - #134
Merged
Merged
Conversation
The token threshold compared prompt_tokens, which under prompt caching counts only the uncached suffix of the prompt, so long cached conversations crossed the threshold without checkpointing. The trigger uses context_tokens (uncached input plus cache reads and writes), as the session host does; checkpoint payloads and markers carry both values.
skhaz
approved these changes
Nov 3, 2026
The stub summarizer records the context_tokens it receives and the checkpoint tests assert it next to prompt_tokens. The default summarizer prompt states the context size, and the stub model takes cached prompt tokens as an argument.
Contributor
Author
…ext-tokens # Conflicts: # src/node/agent/stub/recovery_generate.lua
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
The agent node's checkpoint threshold compared the last action's
prompt_tokens. With prompt caching (the history_tail marker from #90),prompt_tokenscounts only the uncached suffix of the prompt, so a long conversation whose prefix is cached stays far below the threshold and never checkpoints, until the model call fails on the context window. The session host moved tocontext_tokensfor the same reason.Change
tokens.context_tokens(uncached input plus cache reads and writes), which the llm library sets on every normalized response since 0.5.1 (the module already requires >= 0.5.3).refs, the function fallback payload, and the checkpoint marker metadata carrycontext_tokens/checkpoint_at_context_tokensnext to the unchangedprompt_tokens/checkpoint_at_prompt_tokens, so custom summarizers keep their inputs.context_tokensnext toprompt_tokens.cache_read_tokens(scenario input) to exercise cached prompts; the stub summarizer records thecontext_tokensit receives.Tests
threshold counts cached prompt tokens: 200 uncached + 5000 cached tokens against a 1000-token threshold must checkpoint and record 5200 context / 200 prompt tokens. It fails on master (no checkpoint).context_tokensis asserted (5200 in the cached case, 500 in the uncached threshold case). The binding path buildsrefsfrom the same values; the suite has no successful checkpoint binding stub to observe it.make test-sqlite) passes, lint clean.