Skip to content

fix(store): split sibling fields back out when they leak into content - #45

Merged
DottytheHomeless merged 2 commits into
mainfrom
fix/store-tag-leak
Sep 23, 2026
Merged

DottytheHomeless merged 2 commits into
mainfrom
fix/store-tag-leak

Conversation

@MXAntian

Copy link
Copy Markdown
Contributor

Why

A long store_memory argument closed with the wrong tag — </content> instead of </parameter>, or a bare <summary> — makes the tool-call parser keep reading. The sibling fields end up inside content (or summary) as raw XML:

…body text.</content>
<parameter name="summary">the summary</parameter>
<parameter name="importance">9</parameter>

and the real summary / importance / category arrive empty, so zod fills in its defaults. The write succeeds, the row has no summary line, and an importance-9 decision is stored as a 6. Nothing reports it.

A full scan of one production store turned up 47 live rows like this, spread over five months — despite an agent-side instruction to close tags carefully that was in place the whole time. Some were on the summary side (…</summary><parameter name="importance">7), and three followed a correct </parameter>, so "be careful with tags" was never going to be enough.

What

tag-leak-repair.mjs — a pure function run at the top of the store_memory handler, before quarantine routing or the write.

Detection is structural, not substring: a wrong closer immediately followed by a field tag, and the whole tail has to parse as field tags to the end of the string. That distinction matters because the rows most likely to contain these strings are the ones describing the bug:

input result
body</content><parameter name="summary">S repaired — content body, summary S
…a tail like `</content><parameter name="summary">…` then more prose untouched (backticked, and prose follows)
…</content>, which then corrupts importance untouched (no field tag follows)
x</content><parameter name="summary">S</parameter> and more text untouched, reported as ⚠️ possible leak

Merge rule. For fields zod defaults when they're absent (importance, category, memory_type, memory_level), the passed value is the default standing in for the one that leaked, so the leaked value wins. Fields with no default (summary, tags, supersedes, …) keep a non-empty passed value. Invalid values (unknown category, importance 42) are dropped rather than applied.

Feedback. The response gains one line:

Stored memory (id: 1, importance: 9, type: long_term, level: semi_abstract)
🩹 close-tag leak repaired: summary, importance, category moved back out of content — a field was closed with the wrong tag; each <parameter> must end with </parameter>

Tests

  • tag-leak-repair.test.mjs: the parser against leak shapes taken from real corrupted rows, plus prose that must pass untouched
  • tag-leak.integration.test.mjs: spins up the HTTP server on a temp DB, sends a leaked write and a prose write through store_memory, and checks the stored rows
  • Both are wired into CI

Existing rows aren't migrated here. The same function can be run over a store as a one-off (dry-run first, and clear content_vector on rows whose content changed so they get re-embedded).

🤖 Generated with Claude Code

MXAntian and others added 2 commits September 23, 2026 12:07
When a caller closes a long `content` argument with the wrong tag
(`</content>`, a bare `<summary>`, and in a few cases even after a correct
`</parameter>`), the tool-call parser keeps reading: summary / importance /
category / tags land inside `content` or `summary` as raw XML, and the real
fields fall back to their zod defaults (importance 6, category general).
The row looks fine and ranks wrong forever.

An instruction telling agents to close tags carefully held for months and
the leak kept happening, so the repair now lives at the write path.

- tag-leak-repair.mjs: structural detection — a wrong closer followed by a
  field tag, AND the whole tail must parse as field tags to the end of the
  string. Backticked mentions and closers followed by prose pass untouched,
  so a memory that describes this bug is stored verbatim.
- Merge rule: fields zod defaults when absent (importance, category,
  memory_type, memory_level) take the leaked value; summary / tags /
  supersedes only fill in when the passed value is empty.
- store_memory reports 🩹 with the fields it moved back; a leak-shaped tail
  that doesn't parse cleanly gets ⚠️ and no data change.
- Unit test for the parser, integration test through the MCP handler; both
  wired into CI.

Co-Authored-By: 千夏 <qianxia@clawgamers.com>
Three problems found in review, each reproduced before fixing:

- False positives on markup. The tail parser accepted any closing tag, so
  content ending in an Atom entry or HTML fragment (`<content>…</content>
  <summary>…</summary></entry>`) parsed as a leak and was truncated. Now:
  only field / `parameter` closers and the tool-call envelope are accepted
  in a tail; a closer that closes an element opened earlier in the text is
  markup; and text is only cut when at least one valid field is recovered
  and something of the original is left.
- Explicit values overridden. A leaked importance/category replaced the
  passed value unconditionally. It now only replaces an empty value or one
  equal to the zod default; when content and summary leak the same field,
  the content-side value wins.
- Quadratic scan. Every leak-shaped start re-parsed the whole remaining
  tail with fresh slices; 312 KB of repetitive near-leak took 11 s and
  blocked the event loop before quarantine routing. Tail tokens now match
  in place with sticky regexes, and a failed parse skips later starts that
  would hit the same token: the same input takes ~4 ms.

Tests: markup cases, explicit-value survival, empty head, content/summary
conflict, the remaining field types, a size/time bound, and the quarantine
path end to end.

Co-Authored-By: 千夏 <qianxia@clawgamers.com>
@MXAntian
MXAntian marked this pull request as ready for review September 23, 2026 04:25
@DottytheHomeless
DottytheHomeless merged commit 293dd7a into main Sep 23, 2026
2 checks passed
@DottytheHomeless
DottytheHomeless deleted the fix/store-tag-leak branch September 23, 2026 04:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants