Skip to content

Follow-ups to #105 and #106: walk-cache shared values, never evict a search in use - #107

Merged
shuv1337 merged 2 commits into
v2from
fix/105-106-followups
Nov 19, 2026
Merged

shuv1337 merged 2 commits into
v2from
fix/105-106-followups

Conversation

@shuv1337

Copy link
Copy Markdown
Owner

Follow-ups to #105 (CodeMode step budget) and #106 (kept search orders), from their reviews. No default changes: EXECUTOR_MCP_MAX_STEPS still defaults to 5,000,000, and no workflow files are touched.

Why

  • fix(mcp): stop execute programs at a CodeMode step budget #105 shared meta. The store walk (rejectCircularInsertion) cached only leaves, objects holding no objects. A shared value made of many small records, such as a meta of 1,000 records attached to every row, was walked record by record on every store: about 500 steps each, 10 million for 20,000 rows. A legitimate program stopped at the budget in about 1 s.
  • Keep a search's order for its later pages, so walking the catalog is no longer quadratic #106 LOWs in tool-search-store.ts:
    • (a) room() ran before the ranking await, and keep() evicted strict LRU after it, so new searches ranking at once evicted searches still being paged. The 30 s idle rule was bypassed.
    • (b) Idle time read the wall clock.
    • (c) A search too large to keep was remembered in the same cache, with its key's bytes. A key past 6 MB evicted every kept search.
    • (d) It wasn't documented that walks go uncached once 32 orders are each read within 30 s.

What

Store walk (codemode patch, references.js). It now remembers every object whose whole graph it checked, not only leaves.

  • Invariant: every data object inside a remembered object is remembered too. The walk removes the container from the set first, so the container is never inside one, and the walk can stop at any remembered object.
  • A leaf stays remembered until an object is stored into it.
  • Any other object is remembered for the current epoch. Storing an object into a remembered object that another remembered object holds (insideWalked) starts a new epoch, which drops all non-leaf entries at once.
  • A store into an object no remembered object holds, such as the array rows are pushed into, keeps the epoch.
  • A program that mutates the shared value between stores still has it re-walked each time. That's the same cost as before this PR, and it's documented.

Kept orders.

  • admit() decides after the ranking, with the order in hand. It evicts only searches unread for keptIdleMillis; otherwise the order isn't kept. A hit only touch()es; it never evicts.
  • Idle times come from Clock.monotonicTimeNanos.
  • Searches too large to keep live in their own oversized map, bounded to 32 entries and 2 MB of keys, so they never evict a kept order. A search whose key alone exceeds keptOrderBytes is neither kept nor remembered: each of its pages ranks.
  • The keptIdleMillis doc now says that once 32 searches are each read within 30 s, or their orders fill 6 MB, further walks go uncached until one idles.

Proof

All runs were on shuvdev (Linux x64) with Node 24.22.2 and the Bun self-host e2e server.

  • bun run check: passes (format, lint, typecheck of 9 projects, no-unit-tests, e2e boundary).
  • bun run e2e:self-host --test-name "Hosted MCP execute stops a program|An MCP search without a query pages|A search being paged keeps": 3 passed (3).
    ✓ mcp-search-kept-orders.spec.ts > A search being paged keeps its order beside 40 new searches at once and beside a search too large to keep  10999ms
    ✓ mcp-search-walk.spec.ts > An MCP search without a query pages through 8,000 tools ranking them once, each later page reading only its own rows  14289ms
    ✓ mcp-execute-step-budget.spec.ts > Hosted MCP execute stops a program ... complete  49624ms
    Tests  3 passed (3)
    
    The kept-orders scenario also passed 3 of 3 runs on its own.
  • The new assertions fail on the old code:
  • New e2e cases:
    • mcp-execute-step-budget: 20,000 rows each pushed with a shared 1,000-record meta complete in under 5 s. Storing a row into that shared value is still rejected as a circular value; this case fails when the epoch isn't advanced. The runaway, native, operator, concat and template cases still stop with TimeoutExceeded.
    • mcp-search-kept-orders (new):
      • (c) A search's second page is kept. Then a search whose key is about 4 M characters pages twice, ranked both times, and the first search's third page is still kept.
      • (a) A search's second page is kept. Then 40 new searches reach their second page in 40 concurrent executes, and the first search's third page is still kept.
  • Soundness of the walk cache. A differential fuzz (scratch, not in the repo) calls rejectCircularInsertion against a fresh reachability check. Over 8 seeds it ran 3.9 M stores and 1.85 M cycles with 0 mismatches. Mutants fail at once: no epoch bump, no insideWalked mark, or the epoch check ignored.

Perf

Step-budget e2e, Bun self-host server, ms:

case v2 (31beaf3) this PR
20k rows × shared 1,000-record meta TimeoutExceeded, 1,317 283
20k rows × shared 50k-column meta 779
runaway while(true) TimeoutExceeded TimeoutExceeded, 7,058
group-by 10 / 50 (Map) 1,691 / 1,332
group rows g[k] = g[k] || [] 1,911

In-process Node 24, CodeMode.execute with 5 M steps, ms:

case v2 this PR
rows.push({ id, meta }) × 20k, 1,000-record meta TimeoutExceeded, 1,095 232
r.meta = meta × 20k TimeoutExceeded, 1,084 278
rows[i] = r after r.meta = meta TimeoutExceeded, 1,014 266
mutating meta.records[k] before each push (documented) TimeoutExceeded, 1,013 TimeoutExceeded, 1,063
while (true) n++ TimeoutExceeded, 1,808 TimeoutExceeded, 1,736
group 50k rows, or-push 1,347 1,324

mcp-search-walk is unchanged: 3,539 ms for the first walk, 473 ms for the repeat.

Not in this PR (backlog)

  • o.x++ on big values
  • Object.fromEntries
  • non-string arguments to string methods
  • ReDoS
  • No e2e covers (b), the monotonic clock: the e2e server's wall clock can't be stepped. The change swaps Clock.currentTimeMillis for Clock.monotonicTimeNanos in the two idle reads.

No prod deploy: that needs a separate sign-off.

mcp-execute-step-budget.spec.ts
mcp-search-kept-orders.spec.ts
mcp-search-walk.spec.ts
mcp-search-statements.spec.ts
mcp-search-concise.spec.ts

A shared value of many small records, such as a meta object of 1,000
records attached to every row, was walked record by record on every
store: about 500 steps each, 10 million for 20,000 rows, so the program
stopped at the 5,000,000-step budget.

The store walk now remembers every object whose whole graph it checked.
A leaf stays remembered until an object is stored into it; any other
object is remembered for the current epoch, and storing an object into a
remembered object that another remembered object holds starts a new
epoch. A differential fuzz of 3.9 million stores against a fresh
reachability check agrees on all of them and fails at once when the
epoch is not advanced.

mcp-execute-step-budget: 20,000 rows sharing a 1,000-record meta now
complete in 283 ms (TimeoutExceeded at 1,317 ms before); storing a row
into that shared value is still rejected as a cycle; runaways still stop
with TimeoutExceeded.
…part

Kept search orders (#106) had three gaps:
- room was checked before the ranking await and eviction after it was
  strict LRU, so new searches ranking at once evicted searches still in
  use; an order is now admitted after the await, evicting only searches
  unread for 30 s, or not kept;
- idle time read the wall clock; it now reads the monotonic clock;
- a search too large to keep was remembered in the same cache with its
  key's bytes, and a key past 6 MB evicted every kept search; such
  searches are now remembered in their own bounded map, and a key too
  large to keep is never remembered.

mcp-search-kept-orders covers the first and third: both assertions fail
on the previous store and pass on this one.
@shuv1337
shuv1337 merged commit 70c9d01 into v2 Nov 19, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant