Skip to content

feat!: establish runtime ownership and durable application history - #28

Merged
GCdePaula merged 32 commits into
mainfrom
feature/review-ledger-and-tracks
Sep 11, 2026
Merged

feat!: establish runtime ownership and durable application history#28
GCdePaula merged 32 commits into
mainfrom
feature/review-ledger-and-tracks

Conversation

@GCdePaula

@GCdePaula GCdePaula commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

The sequencer must stop issuing soft confirmations after a diagnosed terminal fault and recover from durable facts before serving again. This change makes that boundary explicit, simplifies startup recovery, and gives native applications a direct ownership contract without changing canonical transaction ordering.

Changes

  • Prepare runtime resources before admission, then launch workers together. Retain exclusive data-directory ownership through blocking work. Diagnosed terminal runtime faults abort immediately; ordinary shutdown and recovery exits still drain, including in-flight reader appends.
  • Derive admission from the process lock, completed setup, and canonical-divergence marker. Express startup recovery as an ordered procedure, preserving flush → safe-head sync → cascade and final admission checks. Terminal-fault records remain diagnostic only.
  • Establish durable era/generation metadata, application-history bases, execution offsets, and snapshot/replay consistency checks. SQLite owns execution-offset contiguity. The public feed still uses physical rowids; the history-version API remains separate work.
  • Make Application own and report progress by value. Shared execution verifies successful count/clock transitions. Distinguish validation rejection from fatal engine failure; retain Send and remove host Clone + Sync requirements.
  • Permit mutable checkpoint creation while requiring durable immutable dumps, independent restores, and unchanged logical state. Checkpoint paths may be files or directories; recovery state and canonical comparison bytes may differ. Move canonical inspection to the optional CanonicalState trait.
  • Simplify lane outcomes and its drain cursor, and unify atomic drain/promotion storage. Preserve ordering, attempt limits, commit-before-ACK, and frame policy. Restrict browser CORS to ingress and align watchdog tooling with Lua 5.4.
  • Bootstrap the fee quote during setup. Runtime refresh connects lazily and retains the persisted price through transient quote failures.

Breaking changes

  • Database: the prototype baseline schema changes; there is no in-place upgrade for older databases. Use a fresh data directory and the supported setup/checkpoint-recovery procedure.
  • Supervisor: terminal runtime faults now terminate with SIGABRT. Treat it, like exit 30, as requiring intervention rather than automatic restart.
  • Library consumers: command/config/error APIs move from runtime into commands; L1Config moves into l1, and CommandError replaces RunError.
  • Application adapters: follow the migration instructions. This interface change preserves current wallet snapshot bytes, transaction encoding, and rejection semantics.

Validation and review

Local workspace checks, formatting, strict Clippy, 697 Rust tests, and 62 watchdog tests passed. Restart/replay E2E passed. The local recovery E2E reached the resumed finalized snapshot but could not complete watchdog initialization because the installed Lua binding expected a newer Cartesi Machine archive.

Full CI passed, including Rust checks/tests, canonical guest tests, rollups/watchdog E2E, and watchdog Docker smoke tests. That run tested 35596a4; its Git tree is identical to the pushed 8c58d4a after commit rewording and squashing. The new CI run for the rewritten head is in progress.

The reference C bridge and its fallible genesis-factory API are on a separate integration branch. Private DEX engine/scheduler conformance remains unverified; scheduler injection and the public history-version/feed API remain follow-up work.

Suggested review order: Application contract, invariants, runtime authority, recovery, then their implementation and tests.

@GCdePaula GCdePaula changed the title Review ledger and tracks Fix storage decode policy Jul 30, 2026
@GCdePaula
GCdePaula force-pushed the feature/review-ledger-and-tracks branch 2 times, most recently from a8c23eb to 5233880 Compare August 24, 2026 12:00
@GCdePaula GCdePaula changed the title Fix storage decode policy feat!: the runtime authority boundary -- fact-derived admission, containment, and a durable history foundation Sep 7, 2026
@GCdePaula
GCdePaula force-pushed the feature/review-ledger-and-tracks branch 2 times, most recently from 35596a4 to 8c58d4a Compare September 9, 2026 20:42
@GCdePaula
GCdePaula requested a review from stephenctw September 9, 2026 20:42
@GCdePaula GCdePaula changed the title feat!: the runtime authority boundary -- fact-derived admission, containment, and a durable history foundation feat!: establish runtime ownership and durable application history Sep 9, 2026
…story foundation

One process, one data directory, one way in: this lands the authority
boundary the 2026-08 ADR designed, with admission derived from durable
facts and history given real coordinates.

Process ownership and containment:
- Kernel-enforced one-process-per-data-dir (runtime/process_lock.rs); the
  controller retains the lock through settlement, nested blocking work
  retains clones until it actually stops.
- RuntimeScope: the runtime-lifetime capability (sticky containment
  OnceLock, independent two-second terminal-abort watchdog, best-effort
  fault recorder), constructible only from a held lock.
- The Authorized externalization token: acks, L1 sends, WS frames, and
  snapshot-stream starts require a token minted by consulting containment
  — forgetting the check is a compile error, not a convention.

Commands and the exit-code contract:
- commands/ owns the operator brackets (run + worker supervisor, setup +
  fill, flush); runtime/ is the capability substrate consumed crate-wide.
- One CommandError taxonomy with the R4 exit projection (10 expect
  recovery / 20 transient / 30 terminal, do not restart, page / 40 setup
  needs recovery); terminality declared beside each worker error type.

Fact-derived admission (reviews L2/L3):
- Admission is three facts, each with one owner: the process lock,
  two-sided setup_complete, and the absorbing canonical_divergence marker.
  No lifecycle state machine, no operator acknowledgement; standard
  recovery is automatic and cockroach rebuild is the one manual path.
- The only durable telemetry is the terminal_faults black box: append-only
  terminal-cause rows, written best-effort and verdict-neutrally.
- Boot is prepare -> admit (the same pure reducer re-run over one
  consistent fact set) -> single-use AdmittedRuntime -> non-yielding
  launch.

Startup recovery:
- A pure reducer over one transactionally consistent inspection selects at
  most one phase; every completed phase returns to inspection; flush/sync
  witnesses are boot-local and never persist.
- admission.tla proves terminal dominance, one-phase-per-inspection,
  witness requirements, crash/restart soundness, and capability soundness
  (TLC: 860 generated / 266 distinct / depth 13 / 0 violations).

Durable history foundation:
- (EraId, RecoveryGeneration, K) coordinates, each recorded at the moment
  it happens; the executed_inputs canonical projection; setup completion
  as a synchronous=FULL linearization point; snapshot lifecycle hardening
  (lease reset, GC, orphan sweep, promotion restamp).
- sequencer-core: one shared typed execution boundary writes the consensus
  coordinate; raw hooks are structurally unreachable; app-boundary decode
  and genesis paths return typed refusals.
The documentation practice gains a lifecycle, now recorded in AGENTS.md
("Documentation Practice"): living docs are timeless — present tense,
reasoning inline, no dates, no amendment banners, no review codenames —
while history lives only in docs/review/ and commit messages, and a review
ledger is distilled when it closes. Conclusions with reasoning outlive the
path taken to them. Plus the comment rule: comment the non-obvious, never
restate what the code expresses.

- docs/review/register.md is the new hub: open findings (statuses verified
  against the tree), owed tests with recipes and harness levers, open
  maintainer decisions, settled decisions with each reasoning's current
  home, refuted proposals (do-not-re-propose), and a historical codename
  map so older commit messages stay decodable. CLAUDE.md/AGENTS.md point
  there.
- All eight dated ledgers distilled to stubs or compact decision records
  (docs/review/ 2,392 -> 502 lines); full originals remain in git history.
- The authority-boundary ADR rewritten present-tense (769 -> ~210):
  context, the four mechanisms, rejected alternatives with their
  arguments, revisit conditions; cutover chronology and raw benchmark
  tables reduced to their conclusions. The superseded terminal-containment
  plan is deleted; the coordination-tracks doc is rewritten to current
  truth (Track 5's LUT rationale kept in full).
- Living docs scrubbed of tense leakage; the concepts formerly cited as
  review codes now have real names homed in invariants.md, which also
  gains the do-not-simplify list and I9's content-equal => effect-equal
  argument. ~90 codename citations removed from code comments and the
  schema.
- Adversarial review of the distillation (24 agents, three lenses):
  23 confirmed findings fixed. The lost-content lens restored two real
  casualties with updated homes: cockroach recovery's flush is best-effort
  by construction (no watermark survives the wipe; the fail-safe
  flush-floor option recorded) -> cockroach.md step 2 + register; and the
  supervisor recipes (systemd RestartPreventExitStatus; on Kubernetes the
  crash-loop bound is exit-code alerting, since Deployments cannot honor
  exit codes and there is no boot gate) -> the operator runbook.
L2TxFeedConfig::default() set batch_submitter_address to None — "filter
nothing", the I11-violating value — so the feed a test or future caller
got by default fanned our own batch envelopes out to WS subscribers.
Production was correct only because the runtime glue remembered to
override the default.

The address is now a required constructor argument and the Option is
threaded out end to end: the filter compares plain addresses (matching
the inclusion lane's spelling), and the unfiltered SQL arm in
storage/egress.rs — reachable only through the deleted default — is
gone. Fixtures that seed no own-batch rows pass a sentinel address that
collides with no seeded sender; Address::ZERO is a real fixture sender
and must not be used for that.
…he launch bracket

Simplification pass over the run-command startup glue, adversarially
reviewed: two verification rounds over the proposals and a fidelity
review over the landed diff. Behavior-preserving; operator-visible
deltas are limited to log fields (the "listening" line reports the
resolved bound address; drain warnings carry a structured `phase`).

- L1Config carries the pinned DeploymentIdentity verbatim, built once in
  run(): exactly one route to identity below the gate. The poster's
  start_block round-trip through the reader is gone, and the test
  fixture builds one identity literal instead of hand-copied fields.
- PreparedRuntime is now exactly the launch-argument bundle: config
  structs die in prepare (lane_config, api_config, snapshot_state,
  bound_addr in; run_config, l1_config, db_path, dumps_dir out), and
  launch is six spawns plus the "listening" log.
- The EIP-712 domain is derived in prepare from l1_config.identity —
  one source, now that the identity travels whole.
- ApiConfig owns the deployment-varying ingress values (domain, payload
  bound) via a mandatory constructor; the service limits stay module
  constants, documented as deliberately not operator-tunable.
  start_on_listener drops to six parameters.
- AdmittedRuntime is collapsed into launch(self, RuntimeAdmission): the
  witness is the mechanism, the wrapper was presentation. Vocabulary
  swept through AGENTS.md, the ADR, invariants, the recovery README and
  admission.tla comments; the spec's admittedRuntime state names are
  deliberately kept (TLC unchanged: 860/266/depth 13/0).
- finish's two drain loops merge into one two-phase loop with a single
  precedence match (contained > primary > signal > first drain error).
  The register's drain-merge refutation was scoped to the naive
  re-awaiting sketch ("as sketched" in the 2026-08-18 source); the
  entry is scope-narrowed with the source cited.
- select_first_exit destructures Self exhaustively, closing the one
  per-worker site a seventh worker could silently skip.
- Startup snapshot hygiene moves to commands/run/startup_hygiene.rs;
  its five order-critical steps were mis-documented as four.
- Fee-oracle bootstrap: RunFeeOracleBootstrapError deleted (exit codes
  proven unchanged; double-prefixed misconfig messages fixed); the
  transient connect arm reuses the provider it already built. The
  previously untested Some-limb drain path gets a pin.
- The WORKER_LAUNCH_COUNT probe and its global test mutex are deleted:
  the ProcessLock::acquire assertion is the stronger behavioral
  detector, and the tests it serialized are now independent.
The submitter's signing key rode through three Debug-derived structs
(KeyArgs inside RunConfig/FlushConfig, and L1Config) in plaintext hex —
one future ?config log away from leaking. It now enters the process as
SubmitterKey at the clap value_parser: Debug prints [redacted], no
Display exists (a %key log line is a compile error), and the raw hex is
reachable only through expose_secret(), whose call sites are the
provider builders and address derivation. A test pins that a carrier
struct's derived Debug never prints the secret.

No public-address accessor: the key's public identity is the pinned
identity.batch_submitter_address beside it, already in the startup log.
Closes the register's Debug-derive open finding; the startup log's full
RPC URL (token-bearing per the help-leak test's threat model) is a
recorded, deferred tension.
Require and persist a fresh fee quote during setup, then retain the last price across transient runtime failures.

Keep the process lock through cancellable feed preparation and make E2E mining follow the safe-block clock without synthesizing future L1 time.
Couple stopped-process outage time with L1 progress, update the backward-clock scenario to assert retryable admission denial, and keep the live stuck-Tip injection recoverable after its detector exit.
Add the dated ledger for the pre-merge stock-take of this branch: the
verdict (proportionate; three residue pockets), the facts verified
first-hand, the eight jury-confirmed and ten jury-refuted simplification
proposals with their evidence, and the fifty-four proposals no jury
examined, marked as such. The register gains findings 19–31, two owed-test
entries, a 2026-09-03 refuted block with an evidence line per entry, and
two hygiene fixes the audit found in the register itself: the
fee-price-age refutation is re-homed under the commit that made it, and
the boot-gate refutation's wording is narrowed to the argument it rests on.
… the log line

The rollups-e2e job was red on `aging_open_tip_runtime_danger_zone_exit_test`
for a deterministic reason: the assertion grepped the child's log for
`status=TipInDanger(`, but tracing-subscriber styles field names and the
`=` with ANSI escapes whenever `NO_COLOR` is unset (no TTY check), and the
harness passes the parent environment through. It passed only in shells that
export `NO_COLOR`.

Both wallet-sequencer binaries now enable ANSI only when stdout is a
terminal — the production fix: a daemon writing to a pipe, a file, or
journald must not interleave escape codes into its operator log.

The scenario asserts the exit code instead: 10 is the observed-danger class
(`TipInDanger` / `ClosedBatchInDanger`), a clock fallback would exit 20, and
the scenario has no closed batch — the same discrimination the log grep was
reaching for, on a documented contract rather than a rendered line. Verified
locally: the scenario passes and the log file carries no escape bytes.
Four independent reviews found the same defect class on this branch: comments
that claim more than the code enforces. In a codebase where comments are the
audit surface, an overstated guarantee is a missing check.

- `authorize()`, the ADR, and the register's settled entry claimed four
  compile-forced externalization primitives; three functions take the token
  (ack, L1 send, WS emit). The snapshot-stream start, the `POST /tx` success
  body, and the lane's two mutation commits are hand-placed consults bounded
  by the exit contract, and now say so.
- `Authorized`'s doc claimed a consult "at this effect boundary"; the token is
  `Copy` and lives for the scope borrow, so it proves a consult at some point
  in that borrow — the ADR's bounded lag, stated where a reader meets it.
- The flush witness was called "non-clone" in the recovery README, the TLA
  header, and a test comment; `RecoveryProgress` derives `Copy`. The property
  is that it is boot-local.
- The process-lock witness comment said a live witness means runtime work
  has not drained; it also covers the controller's own clone through
  settlement.
- Three "journal" sites missed by the black-box rename, one dated rename
  history and a removed "acknowledge" command in the error module's header,
  a module doc arguing with a hypothetical simplifier, and a recovery README
  naming a `DangerDetectorExit` type that does not exist.
- `preparation_outliving_clean_facts_cannot_launch` never calls `launch`; it
  asserts that final admission refuses over aged facts, and is now named so.
AGENTS.md's documentation practice keeps review-item codenames out of code
comments and living docs; the distillation commit removed ~90 citations and
missed fifteen. Each is replaced by the reason itself or by the invariant id
it stood for (R1a → I14, R2 → I9/I15, "review R4 class 10" → the exit-code
class), or by the plain word "regression" where the code was only a label.
Three of the fifteen were added by this branch (L2, H6, D10); the rest
predate it.
Five `Storage` methods were `pub` with only test callers, one of them an
unguarded batch-tree mutation sitting beside its guarded replacement:
`ensure_open_tip` (production opens the Tip through the reducer's guarded
`EnsureOpenTip` phase), `close_frame_and_batch` (production closes through
the pending-snapshot twin, I7), `latest_batch_index`,
`ordered_l2_txs_for_batch`, and `promote_finalized` (production promotes
inside the drain transaction, I6). All five are now `#[cfg(test)] pub(crate)`;
their intra-doc links and the snapshot lifecycle doc name the production
paths instead. Gated rather than deleted: the two index helpers have callers
across three test modules.

This is the benefit the in-crate integration-test move was meant to unlock.
The finality wait timing out and resubmitting is the mempool flush's ordinary
rhythm on a slow chain, not a fault; it was logged at `error!`, so every
healthy recovery boot paged.
The comment claimed a `debug_assert` that does not exist. The high limbs are
dropped without a check; that is sound because every caller's inputs are
table entries or partial products for an exponent already bounded by
`MAX_EXPONENT` in `fee_to_linear_fixed`, and partial products never exceed
the final value. The contract half of the fee-determinism finding (the
LSB-first floor order) stays open.
…body

The one `POST /tx` 500 path that still echoed internals — an application
`Internal { reason }` or I/O error during execution — now answers with the
fixed "application internal error" the other internal paths already use.
The reason still travels on the lane's `ExecuteUserOp` error and the log,
which the test now pins alongside the fixed client message.
`ApplicationProgress::new` panicked on an incoherent pair and had only test
callers; its own doc said to use `try_new` on the one path that constructs
a pair from data. `try_new` is now the single constructor (a genesis
instance starts from `Default`), tests use it with `expect`, and the
should-panic test became a direct assertion on the fallible form.
The ledger gains a "Landed" section so a later session resumes from the tree.
The register's findings keep stable numbers: closed entries (4, 5, 17, 21,
25, the comment half of 8, the gating half of 10) are reduced to one-line
closure notes rather than deleted, so the ledger's citations stay valid.
…ites the black box once

`RuntimeScope` loses the in-scope fault recorder: the `FaultRecorder`
alias, the `OnceLock` field, `set_fault_recorder`, the recorder call inside
containment, and its installer in `PreparedRuntime::prepare`. Containment is
now three non-blocking steps — CAS-elect the cause, arm the two-second abort
watchdog, request shutdown — and the "arm the watchdog before recording,
either may block" ordering hazard goes with the second SQLite writer that
caused it. The command bracket's settlement write (`record_terminal_fault_best_effort`,
which already existed as the second of two writers) is the black box's one
writer, so a contained run records one row, not two.

The accepted loss, stated in the runbook where an operator reads it: any
death that does not return through its bracket — an abort at the two-second
deadline, a controller panic, SIGKILL — leaves only the process logs, and
the single write has no second attempt. The row is telemetry; restart policy
is the exit code, which is untouched.

`run` gains a verdict-neutral startup read: `warn_on_previous_terminal_fault`
logs the latest black-box row once, ahead of the admission preflight so a
refused boot still says why the last one died; nothing branches on it, and a
missing or unreadable table is a debug line. The black box thereby gets its
first in-product reader.

Tests: the two recorder-only tests are deleted; containment's first-reporter
election is pinned without a callback, and the settlement write is pinned
directly (a transient verdict leaves no row; a terminal one lands the
error's display form). The invariants check policy, AGENTS.md, the ADR, the
recovery README, the admission model's comments, and the runbook are
restated to match. Closes register finding 24.
…n-optionally

`acquire_finalized_lease` returns `FinalizedLease { inclusion_block: u64,
dump: LeasedDump }`; `LeasedDump` no longer carries an `Option<u64>` that was
`Some` for one query and `None` for the other. `finalized_state` destructures
the lease, and its branch on an impossible `None` — which escalated a type
artifact to terminal containment, against the check policy's "no
Option-handling for can't-be-None" — is deleted. The column is `NOT NULL`
with a CHECK at the engine, and corrupt-row containment is unchanged (the
persistent-storage classifier and the decode panic, as the existing test
pins). Wire behaviour of `/finalized_state` and `/latest_snapshot` is
unchanged: headers, ETag, the 304 path, the 404, and the guard-after-commit
ordering. Closes register finding 26.
…its postcondition

`ensure_open_tip_for_recovery` guarded two conditions with one disjunction,
so an already-open Tip surfaced as `StaleDecision { expected: Safe, actual:
Safe }` — a line the test pinned as intended. The two checks are now
separate: a non-`Safe` danger is still `StaleDecision`; an already-open Tip
is the payload-free `TipAlreadyOpen`, paired with a retry reason so
`classify_mutation` carries it to the operator (retry, exit 20; a restart
inspects, finds the Tip, and admits).

The same phase is the only edge back into `Repaired` without a Tip, so it
is the reducer's one cycle, and no watchdog exists on the boot path. It now
re-reads `has_valid_open_batch` inside its own transaction after opening and
returns `TipMissingAfterOpen` — classified `refuse`, exit 30 — rather than
commit without one. A typed refuse, not a `debug_assert` (compiles out) and
not a retry (would relocate the spin into the supervisor). `drive_recovery`'s
doc records the termination argument: at most five phases per attempt, the
process lock making this the only writer between them. `admission.tla`'s
`hasOpenTip' = TRUE` is now an enforced postcondition, and says so.

A fault-injection test invalidates the new Tip the moment its first frame
lands (invalidating on the batch insert is caught one step earlier by the
schema), so the opener returns `Ok` with no valid open Tip, and asserts the
refuse plus full rollback. Closes register findings 20 and 23.
…on half

`DangerDetector`, `InputReader`, and the fee-oracle worker used their
`RuntimeScope` for exactly one thing — `wait_for_shutdown` — and each already
holds a construction-required `ProcessLock` clone for its data-directory
work. They now take `ShutdownSignal`, the scope's notification half;
`PreparedRuntime::launch` passes `shutdown.signal()` for the three and the
scope to the lane, HTTP server, and submitter, which externalize or contain.

Nothing the scope carried is lost for them: the lock clone they own is a
clone of the same descriptor the watchdog's weak witness observes, so a hung
one still aborts at the deadline; the signal is a clone of the same
notification state containment pokes, so a contained fault still stops
them; and their terminal exits were always classified by the supervisor,
never by worker-local containment. Their tests construct a bare signal
instead of the default test scope, which leaked one temp-dir lock per use.

The prose that claimed every worker held a scope clone is restated where a
reader meets it: the `ShutdownOnDrop` doc, the launch-boundary test comment,
the process-lock module doc, the scope's own rule, and the ADR's mechanism
1. A pre-existing broken intra-doc link in the shutdown module is fixed
while the file is open. Closes register finding 22.
…failure one verdict

A missing, unreadable, or non-text batch-submitter key file now exits 30
like bad key content one call later, instead of restart-looping at 1;
environmental I/O still exits 1. The read returns a typed
`BootstrapError::KeyFile { path, source }` whose message names the path
and never the contents. Both halves are pinned, including the EIO kind
an operator actually hits.

`RecoveryFailure::Provider(String)` carried two verdicts, decided by its
construction site. It is split into `ProviderUnreachable` (retry) and
`SignerMisconfig` (refuse), classified at birth by
`classify_signer_provider` and pinned, payload and polarity, against the
`BootstrapError` projection it must agree with. `classify_input_reader`
now documents what it can actually receive (only the sync's
`create_provider` produces `Bootstrap` here; discovery-time facts are
`setup`-only) and why `Bootstrap` and `Join` flip polarity between the
startup phases and the live worker.

Register: findings 19 and 27 closed; finding 32 opened (the same L1
misconfiguration exits 1 under `setup` and 30 under `run`; recorded, not
fixed); the flatten-`RecoveryError` refutation annotated now that its
two-verdict premise is gone; drifted cites retargeted.
…alues

The five per-class exit-code tests, the verdict test, the two startup-
reader tests, the fee-oracle fatal-math test, and the two app-bootstrap
tests fold into five class functions and one table test that asserts,
for every row, the class and that the black-box verdict agrees with it
(`is_terminal` exactly for class 30). Every shape, reason string, and
rationale comment survives; two rows are new (`ProviderUnreachable`
under retry, `SignerMisconfig` under refuse). A second test pins the
five verdicts to the integers 10/20/30/40/1 through an exhaustive match,
so a renumbered constant or an unnumbered sixth verdict fails here, not
in a runbook.

Two wire-value pins outside the table: `run` on a never-set-up data
directory dispatches to 30 through the real command bracket, offline by
construction (the admission preflight refuses before the key is read or
a provider is built); and the harness gains an opt-in
`stop_expecting_clean_exit` that asserts SIGTERM→0 on a healthy
sequencer, used at the healthy stop of the stale-batches scenario. The
runner's per-scenario shutdown stays lenient on purpose.

Register: the owed exit-code test keeps only its per-variant e2e half;
the `classify_input_reader` polarity pins landed with the previous
commit.
Make AGENTS.md an architecture map and give the protocol contracts,
invariant register, recovery guide, and ADR explicit ownership of their
facts. Align documentation claims with the code and explain non-obvious
behavior in module comments.

Consolidate the review ledgers into the history register and record the
settled decisions and validation from the documentation review.
Rely on SQLite's contiguous-offset trigger as the transactional enforcement
point. Remove the duplicate Rust pre-query and validation loop while
preserving fail-loud behavior for invalid history.
Wait for the current blocking append before returning a clean reader exit.
This lets the worker observe committed divergence and append failures, and
keeps process ownership held until the write has actually stopped.
Replace the phase driver and its mirrored progress state with an explicit
startup sequence. Preserve guarded repairs, flush then post-flush sync before
cascade, retry/refusal classification, and fresh admission after preparation.
Update the admission model and tests around those ordering requirements.
Stop a diagnosed terminal runtime fault immediately. Ordinary operator,
recovery, and operational shutdown still notify workers and drain their work.
Remove the containment tokens and terminal-watchdog machinery whose purpose
was to keep a partially failed runtime alive during that drain.

BREAKING CHANGE: diagnosed terminal runtime faults abort with SIGABRT (usually
shell status 134), without graceful drain or durable command settlement, rather
than returning to an embedding caller. Supervisors must treat this as terminal.
CommandError::StorageInvariantViolation is removed.
…eation

Let the application own its persisted progress and return it by value. Shared
execution preflights overflow and checks the exact successful transition.
Keep Send, remove unused Clone and Sync bounds, and separate canonical
inspection from native execution. Clarify durable, immutable checkpoints and
independent restores for file or directory layouts.

Collapse the inclusion lane's duplicate turn outcomes and drain-range state,
and combine frame closure with optional promotion in the same transaction.
Preserve ordering, rejection semantics, canonical bytes, and commit-before-ACK.

BREAKING CHANGE: Application implementations provide progress() by value;
apply hooks advance it themselves. Execution capabilities and the mutable
progress accessor are removed. Validation returns Result<ValidationOutcome,
AppError>, create_dump takes &mut self, and canonical_snapshot_bytes belongs
to CanonicalState. The generic export_state method is removed.
Scope CORS to POST /tx and use Lua 5.4 consistently across the development,
CI, and watchdog tooling. Preserve the existing Lua executable override.

Record application integration validation and environment limits in the
review notes, and correct the application-progress invariant link.
@GCdePaula
GCdePaula force-pushed the feature/review-ledger-and-tracks branch from 8c58d4a to ef5f640 Compare September 10, 2026 15:31
@GCdePaula
GCdePaula marked this pull request as ready for review September 10, 2026 15:32

@stephenctw stephenctw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@GCdePaula
GCdePaula merged commit 5e3d621 into main Sep 11, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants