Skip to content

feat(agent): keep the request prefix stable across steps - #237

Merged
wolfy-j merged 17 commits into
masterfrom
fix/agent-stable-prefix
Nov 4, 2026
Merged

wolfy-j merged 17 commits into
masterfrom
fix/agent-stable-prefix

Conversation

@wolfy-j

@wolfy-j wolfy-j commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Claude binds thinking blocks and the prompt cache to the exact request prefix: top-level system, tools, and every earlier message. Content rebuilt or moved between requests, while sitting before already produced thinking blocks, invalidates them from that point. With #235 the API drops those blocks instead of returning 400, but the reasoning is then lost on every later request and the cache is rewritten. Sources in the framework:

  • The time_aware trait put the current time (rounded to 15 minutes) into the system prompt.
  • Per-step notes moved: memory recall was a per-step developer message at the tail that hosts persisted at a different position; Claude rendered developer notes differently depending on where the rolling cache marker sat; Bedrock hoisted developer notes into system.
  • After checkpoints and handoffs, thinking blocks bound to an old prefix were replayed with no shared way to remove them.

Change

llm

  • A SYSTEM or DEVELOPER message with metadata.preserve_position = true is a positional note (prompt.is_positional_note). Claude and Bedrock append it as its own text block to the preceding user turn (including a tool-result turn), or open a new user turn when none precedes it; it is never merged into earlier blocks and never placed in top-level system. Google renders it as a user turn; OpenAI and OpenAI-compatible keep it positional.
  • Developer notes on Claude and Bedrock always render as their own text block <developer-instruction>…</developer-instruction> appended to the preceding user turn, independent of cache markers, never edited into earlier blocks. On Claude a note that followed plain user text now sits in its own block instead of being joined with "\n"; notes after a cache marker, an assistant turn or tool results now carry the wrapper too. Bedrock renders developer messages as user content instead of system.
  • builder:drop_stale_thinking(is_current) removes thinking_blocks as a leading run: through the last stale message, refusing (history unchanged) when that would split an assistant turn with an unanswered call. Stripped messages are copies; caller tables are not mutated. An empty list carries nothing.
  • Google's mapper no longer clears metadata on caller messages.
  • Golden request-body tests for all five drivers; Claude and Bedrock developer cases are recorded from this branch, everything else from master.

agent

  • time_aware contributes agent_options.turn_context.time = { timezone, format? } (trait context keys timezone and time_format); wippy.agent:turn_context.render(agent_options, now) renders the note to the exact minute for hosts to persist once per user turn. time_interval is rejected with an error naming the replacement.
  • Trait build-function errors fail compilation (previously swallowed); a failing prompt function fails the step, keeping kind, retryability and details.
  • Memory recall runs whenever the agent has a memory contract: the note is a positional developer note at the tail and is returned in response.context_notes for the host to persist in place. response.memory_prompt is removed.
  • response.prefix_fingerprint: sha256 of the sent Claude/Bedrock prefix projection (system texts incl. hoisted conversation system messages; tools by name, description, schema and native definition), computed before the request. With runtime_options.drop_stale_thinking the agent strips blocks whose stored fingerprint differs (or is absent) before sending; runtime_options.thinking_is_current(msg, fingerprint) lets hosts add their own epoch (e.g. checkpoint id). The fingerprint never reaches the wire.
  • The conversation-tail cache marker precedes trailing developer and positional notes. The sorted-key encoder from attention_read moves into wippy.agent:canonical.

Tests

llm 2083 pass (9 skipped), agent 560 pass (4 skipped); repo-wide tests, lint and manifests pass. Every review finding is pinned by tests that failed before its fix.

Release coordination

  • Merge after fix(llm): let the API drop thinking blocks bound to another conversation #235: the Bedrock developer move and the Claude developer rendering change stored histories' prefix once on upgrade; fix(llm): let the API drop thinking blocks bound to another conversation #235's drop_block turns that into a drop instead of a 400 on bound requests (thinking: always, or adaptive/budget with effort > 0). Unbound requests on enforced accounts can 400 once until hosts strip with drop_stale_thinking.
  • Session and dataflow must persist context_notes, the turn-context note, prefix_fingerprint storage and drop_stale_thinking in the same release; until then they have no time note, and recall still reaches the request but is not persisted.
  • The llm changes release as wippy/llm 0.5.13; the agent requires >=v0.5.13 (earlier llm lacks prompt.is_positional_note). llm and agent publish together. Squash with an explicit commit message without !/BREAKING CHANGE markers so release-please cuts patch releases; the behaviour changes are listed above.
  • Upstream follow-up: go-lua errors.wrap context is not visible from Lua (tostring/:message() return the parent message).

Merge instructions

feat(agent): keep the request prefix stable across steps

Positional notes (system or developer messages with
metadata.preserve_position) render in place on every driver; Claude and
Bedrock developer notes render as their own <developer-instruction>
block independent of cache markers, and Bedrock developer messages are
sent as user content instead of system blocks. builder:drop_stale_thinking
removes thinking blocks bound to another prefix as a leading run.

The agent returns prefix_fingerprint and, with drop_stale_thinking, strips
stale thinking before each request. Memory recall returns its note in context_notes for hosts to persist in
place; response.memory_prompt is removed. time_aware contributes agent_options.turn_context instead of
system prompt text and rejects time_interval; turn_context.render renders
the note for hosts. Trait build-function errors fail compilation and a
failing prompt function fails the step. The agent requires wippy/llm
>= 0.5.13.

KickSide added 15 commits November 3, 2026 16:12
…as a leading run

Positional system/developer notes (metadata.preserve_position) render as
text blocks after the preceding user turn on Claude and Bedrock, as user
turns on Google, and stay in place on OpenAI and OpenAI-compatible; they
never enter the top-level system prompt. A positional note without a
preceding user turn is invalid_request. Bedrock developer messages render
as user text instead of system blocks. The Google mapper no longer clears
caller message metadata. builder:bind_thinking strips thinking blocks from
a leading run of non-current messages and rejects middle removal and open
tool rounds. Golden tests pin message-derived bodies on all five drivers.
time_aware contributes agent_options.turn_context instead of prompt text;
hosts render the note with wippy.agent:turn_context and persist it per
user turn. Memory recall runs only for hosts that set
persist_context_notes and is returned as positional context_notes the
host persists before the step output. The tail cache marker skips
trailing positional notes. Each step returns prefix_fingerprint over the
system prompt, tools and model. Sorted serialization moves to
wippy.agent:canonical and is shared with attention_read.
Developer messages render as their own text blocks after the user turn
they follow (or open a user turn after an assistant turn) and never edit
an existing block, so the same history renders identically wherever the
rolling cache marker sits. The Claude golden body changes only for its
developer messages.

bind_thinking becomes drop_stale_thinking: empty block lists carry
nothing, every block up to the last stale message is removed, and an
assistant turn (consecutive assistant and function call messages,
including function_call content parts) with an unanswered call refuses
removal and leaves the history unchanged.
…errors

prefix_fingerprint hashes the system text as drivers send it (compiled
prompt plus hoisted conversation system messages) and each tool's name,
description, schema and native definition; registry ids, registry
metadata and the model are excluded. A failing prompt function fails the
step with its kind and retryability. A failing trait build function fails
compilation; time_aware rejects time_interval naming turn_context.
turn_context.render validates timezone and format and keeps the guidance
sentence. A test guards the llm >= 0.5.13 dependency on
prompt.is_positional_note.
Claude and Bedrock render a positional note at the start of the
conversation or after an assistant or tool call turn as its own user
turn instead of rejecting it, matching persisted checkpoint-resume and
stop-boundary notes. drop_stale_thinking replaces stripped messages with
copies so caller-held message tables keep their blocks. A Bedrock
developer golden case pins the branch rendering.
The prefix fingerprint is computed before generation; with
drop_stale_thinking the agent strips blocks whose stored
metadata.prefix_fingerprint differs (or that thinking_is_current
rejects) from a copy of the history, including checkpoint replacements,
and fails the step on an open tool round. Prompt function errors keep
their details. time_aware reads its layout from time_format so a generic
agent context format key is not taken as a time layout.
Developer messages, positional or not, render on Claude and Bedrock as
their own "<developer-instruction>...</developer-instruction>" text
block after the user turn they follow, restoring the model-visible
wrapper without editing earlier blocks. Positional system notes stay
plain. The unreachable map_messages nil guards in the Claude and Bedrock
handlers are removed. drop_stale_thinking names the unanswered call id
and tool name when it refuses.

BREAKING CHANGE: Claude and Bedrock render developer messages and positional notes differently: each developer message is its own wrapped text block after the user turn it follows (previously merged into the preceding text, or a separate user message after tool results or cache markers), Bedrock developer messages are no longer sent as system blocks, positional notes render in place instead of the top-level system prompt, and builder:drop_stale_thinking replaces the earlier bind_thinking draft. Stored conversations render a different prefix once after upgrading.
The agent calls prompt.is_positional_note, so it requires wippy/llm
>= v0.5.13; the test app lock records 0.5.13, resolved from the local
llm source through the workspace replacement. time_aware rejects a
time_format that is not a non-empty string. Tests cover stale-thinking
stripping of checkpoint replacement histories. The README states that
the fingerprint belongs on the rows that carry thinking blocks.

BREAKING CHANGE: step responses no longer include memory_prompt; memory recall runs only when the host sets persist_context_notes and is returned as positional context_notes the host persists before the step output. The time_aware trait adds no prompt text, declares agent_options.turn_context for hosts to render with wippy.agent:turn_context, and rejects time_interval. A trait build function that returns an error fails compilation, and a failing prompt function fails the step.
The breaking llm changes in this branch release as wippy/llm 0.6.0, so
the agent requires >=v0.6.0.
The llm changes in this branch release as a patch, wippy/llm 0.5.13.
The last assistant turn with no user message after it is the round the
request continues; the provider requires its thinking, so it is never
stripped or consulted and stale blocks there are left to the provider's
drop-on-mismatch handling. Earlier turns are stripped through the last
stale one before it.
…urce lock

Stale-thinking stripping leaves the current tool round intact. The
package source lock records wippy/llm 0.5.13 resolved from the local
llm source, matching the manifest requirement.
… current

drop_stale_thinking scans back over the trailing non-assistant messages;
the last assistant turn stays the current round unless a user message
follows it with no function result in between. Steering messages and the
image message get_messages inserts after a tool result join the
tool-result turn, so the round keeps its thinking.
…ults

Covers stale-thinking stripping when a user message follows the round's
tool results and documents the rule.
drop_stale_thinking counts any message that renders as user content
(user, developer, positional system note) when deciding whether the last
assistant turn is still the current round; hoisted system messages and
cache markers do not.
@wolfy-j
wolfy-j marked this pull request as ready for review November 3, 2026 23:55
@wolfy-j
wolfy-j requested a review from skhaz November 3, 2026 23:55
KickSide added 2 commits November 3, 2026 19:20
A checkpoint replacement history recomputes prefix_fingerprint before
its thinking blocks are judged, so the returned fingerprint matches the
prefix actually sent. The lifecycle contract states that per-step
lifecycle messages change the prefix unless the host persists them in
place as positional notes.
Memory recall runs whenever the agent has a memory contract, as before
this branch. The recall note is sent at the conversation tail as a
positional developer note and returned in context_notes on normal,
truncated and cancelled results for the host to persist in place. The
persist_context_notes runtime option is removed.

BREAKING CHANGE: step responses return the memory recall note in context_notes instead of memory_prompt; hosts persist each context note after the messages they passed and before the step output.
@wolfy-j
wolfy-j merged commit 7d3eae8 into master Nov 4, 2026
25 checks passed
@wolfy-j
wolfy-j deleted the fix/agent-stable-prefix branch November 4, 2026 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant