Skip to content

fix(cache): concurrent writes of one key neither crash the run nor bill twice - #76

Merged
TMHSDigital merged 1 commit into
mainfrom
fix/cache-concurrent-writes
Sep 23, 2026
Merged

TMHSDigital merged 1 commit into
mainfrom
fix/cache-concurrent-writes

Conversation

@TMHSDigital

Copy link
Copy Markdown
Owner

Fixes #27.

The problem. Every write of a cache key went through the same key.tmp. Two workers answering cases with the same text raced on it, and on Windows the loser raised PermissionError [WinError 32] out of Cache.put. That aborted the run and lost every call already paid for: 18 of 20 runs in the review's repro.

What changed

  • Cache.put writes through a temporary file of its own (tempfile.NamedTemporaryFile in the same directory), then os.replace.
  • A write that still fails (a virus scanner holding the file, a full disk) is counted in the cache stats as write_errors and dropped instead of raised. The answer is already paid for, so a missing entry costs one more call on a later run, not this run.
  • The runner holds a lock per cache key across look-up, call and store. Cases that share a key are answered once per run, and the rest are served from the cache instead of each being billed. Different keys never wait on each other.

Checked

  • New tests failed on main and pass now:
    • 16 identical cases at workers=16 make exactly one adapter call, with 15 cache hits and no exception. On main this test reproduced the WinError 32 crash. It passed 20 runs out of 20.
    • With os.replace raising PermissionError, the run still returns every record, counts 5 write_errors, and leaves no .tmp files behind.
  • The review's repro reports 0 crashed runs out of 20.
  • ruff, mypy --strict, and the full pytest suite pass.

🤖 Generated with Claude Code

…ll twice

Every write of a key went through the same key.tmp. Two workers answering
cases with the same text raced on it, and on Windows the loser raised
PermissionError [WinError 32] out of Cache.put, aborting the run and
losing every call already paid for (18 of 20 runs in the review's repro).

Each write now goes through a temporary file of its own, then os.replace.
A write that still fails is counted in the cache stats as write_errors and
dropped, since the answer is already in hand; the cost is one more call on
a later run, not this run.

The runner also holds a lock per cache key across look up, call, and store,
so cases sharing a key are answered once per run and the rest are served
from the cache instead of each being billed. Different keys never wait on
each other.

Fixes #27.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@TMHSDigital
TMHSDigital merged commit 5d23dfa into main Sep 23, 2026
17 checks passed
@TMHSDigital
TMHSDigital deleted the fix/cache-concurrent-writes branch September 23, 2026 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Concurrent cache writes to the same key collide on one .tmp path and abort the run

1 participant