Skip to content

fix: preserve workflow state and runtime resource ownership - #273

Open
KaiEureka wants to merge 73 commits into
mainfrom
fix/major-runtime-audit-20261117
Open

KaiEureka wants to merge 73 commits into
mainfrom
fix/major-runtime-audit-20261117

Conversation

@KaiEureka

@KaiEureka KaiEureka commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Changes

Candidate publication checks the complete patch and preserves affected files and directory metadata before applying it. Recovery replays a failed patch in a scoped directory with the effective worktree attributes. It restores attributable changes and keeps detached original and candidate files reachable so later writes through an open descriptor remain recoverable. The primary recovery directory typically lives in the common Git directory. Its volume records locate original and parked inodes on affected mounts. Restored bytes are staged on the affected mount, and the replay comparison uses the modes that mount can represent. A clean rollback reports the recovery path and removes temporary replay files. Filesystems without hard-link support use equivalent temporary copies. A preparation failure only restores interrupted moves, preserving concurrent edits made before Git applies the patch. A rejected or inconclusive replay retains the recovery files. If recovery finds a conflict or runs out of space, it reports the failure and retains the remaining originals. Structured edits and publication share the same path and ancestor ownership. Docker resolves sibling paths together in each pass, revalidates targets after waiting for a lock, and uses the candidate repository root for publication. Large local snapshots create hard links in one process, with the portable path available when needed. Large recovery commands are staged in bounded fragments, and an interrupted allocation keeps its cleanup owner until its receipt arrives.

Local file tools preserve path components until physical resolution, while remote file tools, search, and authored Workflow source reads use the execution environment's path checks. Native file access retains its existing workspace boundary and descriptor ownership.

Every native provider attempt rechecks the finite budget and cancellation state. Completions and summaries inspect the actual model client's signature before forwarding optional callbacks. Variadic decorators that advertise request revalidation or admission receive the controls they consume, including delegated admission properties. Inherited legacy wrappers preserve the native admission signal through the owned call. Generation timing starts when a slot is acquired, with caller deadlines and cancellation active during preparation and queueing. Existing wrappers settle usage from their responses. Known attempt usage is charged before retry. The final aggregate settles before the request slot is released, and writing the optional ledger happens after release. Authored Workflow children retain reported usage when a budget or cancellation stops a retry. Team inbox delivery retries as individual cleanup owners return enough budget.

Anthropic saves reuse the stored transcript for matching native history entries. The journal compares against the most recently persisted transcript, so a saved edit can be reverted and resumed correctly. Snapshot preparation uses the history already held by the session, including after the parent has stopped admitting child requests. Saved rows remain available while the native view is absent. Transcript reuse and persisted prefix comparisons preserve JSON value types, including booleans inside structured tool inputs. Full native rewrites use the existing atomic checkpoint when a saved generation already exists. A checkpoint can use compact encoding under the same byte limit, or a guarded journal append if checkpoint writing fails. Discovered Workflow modules remain available until runtime finalization and owned cleanup finish, including follow-up work dispatched by completion callbacks. Attached Docker temporary files retain their owner when an atomic publication receipt is interrupted.

Verification

The targeted reproductions exercise native model SDKs, temporary Git repositories, real file operations and real authored Python subprocesses. Docker probes use production shell wrappers with a local transport. They cover restoring 40 of 40 live file-read results, exporting a 34 MiB history, applying a candidate across 3,000 files, preserving concurrent edits and the Git index, releasing a request slot during a slow ledger write, and retaining 1,500 reported tokens on both sides of a stopped child retry. Cleanup tests also run on Python 3.10 and 3.11.

Independent subprocesses open either the original file or the partially applied candidate, then append and flush data after the public workflow has returned. One test removes the linked worktree before the write. The assertions check the original inode, byte contents, warning path, exception type and unchanged Git index. Mounted APFS and ExFAT runs verify that successful adoption cleans up and a partial application retains both inodes after worktree removal. An external deletion during ExFAT preparation remains intact when no patch was applied. A real managed Bash command also overlaps a deletion-only rollback, preserving the ordinary error that permits Duo fallback.

The persisted native view and a journal rewrite both preserve boolean tool input values through save, restore, and a subsequent Anthropic SDK request. The near-64 MiB history completes from a compact checkpoint with its tool receipt intact. A prepublication write failure uses the journal and a postpublication failure preserves the committed generation. The slotted middleware stops before its queued request once only one token remains. On this Mac, the 3,000-file SDK adoption completed in 14.43 and 14.63 seconds with batching, compared with 10.16 and 10.23 seconds before retaining detached files. Each run published all files, preserved the index, and removed the successful transaction's temporary data. The latest local recovery checks passed 123 cases with two skips.

The compatibility update independently reran nested APFS and ExFAT success and
rollback, mixed mounts, late writes, linked checkout removal, cancellation, and
same-device bulk application. The provider controls included direct, transparent,
delegated and inherited wrappers for completions and summaries, with delayed
preparation, queued cancellation, and late usage. The exact-head independent
review covered 54 paired admission probes against main and repeated the mounted
filesystem tests from the committed archive.

Interface compatibility follow-up

A linked worktree can still adopt a patch when its common Git directory is
read-only. Recovery prefers a writable same-filesystem location outside the
checkout, and the tests exercise late writes after linked-checkout removal.
Legacy journal stores preserve their durable append before a checkpoint,
including when the checkpoint fails. Stores that support durable replacement
retain checkpoint-first compaction.

Native routing preserves generation deadlines and accounting for both
ordinary and summary completions. A forwarded admission callback identifies
the actual request when middleware rewrites its messages. Inherited native
controls can reach the native call through an owned scope, preserving
traditional forwarding signatures and keeping auxiliary calls separate.

Independent executable checks compared linked-worktree and journal behavior
against main and the previous PR head. The final provider review exercised
30 paired routing scenarios and 15 original probe cases, including shared
and separate pools, rewritten messages, auxiliary calls, cancellation,
early budget exhaustion and legacy usage reporting.

All 12 GitHub checks passed on 8873348d. Python 3.10 reported 11,081 passed and 80 skipped. Python 3.11 through 3.14 each reported 11,082 passed and 79 skipped. The 3.12 job also ran all 59 Blueprint DOM tests. Its job limit was raised to 45 minutes after two previous runs completed pytest but reached the 40-minute limit before the full job could finish. Full CI

@KaiEureka
KaiEureka marked this pull request as draft November 17, 2026 03:56
@KaiEureka
KaiEureka force-pushed the fix/major-runtime-audit-20261117 branch 3 times, most recently from 9794968 to 760eb42 Compare November 17, 2026 04:16
@KaiEureka
KaiEureka marked this pull request as ready for review November 17, 2026 05:17
@KaiEureka
KaiEureka marked this pull request as draft November 17, 2026 05:32
Hold candidate publication and recovery under the same target and ancestor locks used by structured file edits. Group bridge ownership in one native task and retain one workspace lock for VM edits.

Cover SDK shared agents, native and bridge publication, directory rollback, cancellation, Docker execution wrappers, and VM workspace locking with executable regressions.
Charge known failed-attempt usage before retry backoff and revalidate each physical request against current session and authored workflow limits. Preserve aggregate response, event, and ledger accounting while deducting the usage already charged.

Add loopback TCP regressions for truncated Chat streams, Responses failures, Anthropic retries, shared workflow and team spend, callback failures, and authored subprocess accounting.
Keep callback TimeoutError instances distinct from the Responses round deadline, including unlimited request timeouts. Retain known attempt usage when a callback stops the call.

Cover each callback position with finite and unlimited request timeouts, with provider retry time budgets enabled and disabled.
Resolve original edit paths and alias ancestors before acquiring the complete ordered native lock group. Include physical ancestors while preserving alias nodes, relative execution directories, and symlink parent traversal.

Add SDK publication and cancellation controls with distinct guest and host namespaces, plus native lock ordering and cancelled waiter coverage.
Restore initial request validation and admission notification immediately after acquiring the request slot. Cache the initial output limit for the first provider hook and revalidate subsequent attempts without repeating the first callback.

Exercise provider replacements and native hooks across all three protocols. Keep answer-retention tests focused on admitted retries whose reported input exceeds the estimate and exhausts the budget after returning useful text.
@KaiEureka KaiEureka changed the title fix: preserve candidate recovery and queued request budgets fix: preserve workflow edits and enforce request budgets Nov 17, 2026
@KaiEureka
KaiEureka marked this pull request as ready for review November 17, 2026 08:06
@KaiEureka
KaiEureka marked this pull request as draft November 17, 2026 08:54
Keep the original and parked inodes reachable after rollback so independent writers can finish writing through held descriptors. Report the retained data path and discard replay scratch after a verified rollback.

Exercise late writes across linked worktree removal and preserve ordinary fallback when a failed publication detached only original files.
Prepare same-filesystem replacements before moving originals. On a preparation failure, restore only interrupted moves instead of replaying a patch that never touched the worktree.
Use type-sensitive equality for reusable transcript rows, journal prefixes, and checkpoint equivalence so a resumed Anthropic session preserves structured tool inputs.
Use the installed Python interpreter once for large local patches instead of launching a hard-link process per file. Keep the portable fallback for environments without that interpreter and for filesystems that require the rename-based path.
Keep recovery metadata in the primary directory and retain physical originals and parked outputs on their own mounts. Stage restored copies locally and compare the mode that the live filesystem can materialize before moving a candidate out of the worktree.

Exercise a mixed-device patch, bind-mount identity, new and existing files, and filesystems without hard links through the public workflow API.
Honor admission callbacks consumed by explicit or delegated middleware. When a legacy wrapper reaches the native pool through an inherited capability, carry the deadline start hook in the owned call without passing a new keyword through its signature.

Keep both generation and summary timeouts anchored to admission. Verify queueing, middleware preparation, cancellation, caller deadlines, and late usage while preserving revalidation and legacy forwarding behavior.
Choose a writable recovery directory on the worktree filesystem when common Git metadata is read-only. Prefer a location outside the linked checkout so detached files remain writable after the checkout is removed.
Use checkpoint-first persistence only for stores that support durable snapshot replacement. Retain journal-first autosaves and their established error handling for the existing journal interface.
Preserve admission callbacks that middleware already forwards. Carry inherited native controls through the owned completion when a legacy signature hides them, and match the forwarded request before claiming scoped budget and usage callbacks.

Keep older completion signatures usable while preserving admission deadlines, early budget checks and usage accounting for native routing, compaction and auxiliary provider calls.
@KaiEureka
KaiEureka force-pushed the fix/major-runtime-audit-20261117 branch from f5c3e3d to d2d5716 Compare November 19, 2026 09:25
Two runs reached the 40-minute cap after pytest passed, before the remaining DOM checks could finish. Give only the Python 3.12 matrix job 45 minutes and retain the same checks and the other matrix limits.
Reject conflicting native checkpoints at compacted cursors and preserve the peer snapshot across fallback and terminal saves. Compare string subclasses by their JSON text and retain missing creation timestamps in restored legacy messages.
Match forwarded admission callbacks by identity. Preserve request identity across legacy message copies so warmups and equal-payload helper calls retain their own budgets and usage.
Account provider retries and fallback completions within their own request scope. Keep copied results idempotent and emit usage for every settled provider call.
Compare integer and float subclasses by their encoded values and compare list and dictionary subclasses recursively. Preserve the distinctions between boolean and numeric values, integer and float values, and signed zero.
Settle a native provider with tracked attempts when middleware forwards only the retry callback. Preserve the provider error on usage events when a fallback completes the request.
Deduplicate copied provider usage after middleware redacts raw telemetry. Keep the native request identity out of exported usage fields.
@KaiEureka
KaiEureka force-pushed the fix/major-runtime-audit-20261117 branch from 1e36786 to 5282ba0 Compare November 20, 2026 01:48

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant