-
Notifications
You must be signed in to change notification settings - Fork 0
Docs: state the post-merge version-two status across the documentation #98
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -58,17 +58,27 @@ Keep is required to refuse all three, before mutating anything. | |
| - **Version-2 retention and migration, forward path.** Explicit retention | ||
| roots, deterministic closure verification, a one-way 21-phase migration, | ||
| and a 17-phase retention publication — all with production filesystem | ||
| writers, all preserving every version-1 byte. | ||
| writers, all preserving every version-1 byte. Reopening a migrated store | ||
| jointly admits its marker, intent, and receipt, binds the root's device, | ||
| mount, and inode identity to the intent, and pins the directories it | ||
| admitted. Publication binds this store's own catalog head and the catalog | ||
| it selects, and refuses retained stages, superseded candidates, substituted | ||
| files, replaced protocol directories, and every namespace or capacity | ||
| violation before it writes anything. Each refusal is a typed value, not a | ||
| string. | ||
|
Comment on lines
+67
to
+68
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
For a byte-equal file substitution during publication, the specific refusal is still only a string: AGENTS.md reference: AGENTS.md:L71-L74 Useful? React with 👍 / 👎. |
||
|
|
||
| ## What it does not do yet | ||
|
|
||
| Version 2 writes correctly from a clean start. It cannot yet pick up the | ||
| pieces if it dies partway through. Until it can, **version 1 is the only | ||
| store admitted for production.** | ||
| Version 2 writes correctly from a clean start and, if it finds the residue of | ||
| an interrupted publication, refuses rather than guesses. Nothing yet recovers | ||
| that residue, and readers have no fence, so **an interrupted version-2 | ||
| publication waits for a human until #19 lands.** A version-1 store stays | ||
| admitted until its owner migrates it; migrate only if you accept that wait. | ||
|
Comment on lines
+74
to
+76
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This migration advice mentions only the wait caused by an interrupted publication, but a completely successful migration can also become unavailable after an ordinary reboot, unmount/remount, or restore changes AGENTS.md reference: AGENTS.md:L152-L156 Useful? React with 👍 / 👎. |
||
|
|
||
| | Gap | Tracked | | ||
| | --- | --- | | ||
| | Restart recovery for retention publication and migration | [#19](https://github.com/flyingrobots/keep/issues/19) | | ||
| | Restart-stable root identity coordinate in the migration intent | [#97](https://github.com/flyingrobots/keep/issues/97) | | ||
| | Reader fence binding one consistent catalog + retention snapshot | [#19](https://github.com/flyingrobots/keep/issues/19) | | ||
| | Precise verification reports at explicit depths | [#20](https://github.com/flyingrobots/keep/issues/20) | | ||
| | Garbage collection and identity-preserving compaction | [#21](https://github.com/flyingrobots/keep/issues/21) | | ||
|
|
@@ -208,7 +218,8 @@ has one job; this one is the front door. | |
| | Understand what is proved and what is not | [`docs/invariants/`](docs/invariants/) | | ||
| | Read the byte-level formats | [`docs/formats/`](docs/formats/) | | ||
| | See the architecture and port boundaries | [`docs/architecture/`](docs/architecture/) | | ||
| | Follow the crash and recovery rules | [`segment-store-v1/recovery.md`](docs/formats/segment-store-v1/recovery.md) · [`segment-store-v2/recovery.md`](docs/formats/segment-store-v2/recovery.md) | | ||
| | Follow the crash and recovery rules | [`segment-store-v1/recovery.md`](docs/formats/segment-store-v1/recovery.md) · [`segment-store-v2/recovery.md`](docs/formats/segment-store-v2/recovery.md) · [`segment-store-v2/migration-recovery.md`](docs/formats/segment-store-v2/migration-recovery.md) | | ||
| | See how a retention generation is published | [`segment-store-v2/retention-publication.md`](docs/formats/segment-store-v2/retention-publication.md) | | ||
| | Check reproducible performance evidence | [`docs/benchmarks/`](docs/benchmarks/) | | ||
| | Run the language-neutral corpora | [`conformance/`](conformance/) | | ||
| | See what changed | [`CHANGELOG.md`](CHANGELOG.md) | | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -15,7 +15,7 @@ case is not evidence. | |
| | `KEEP-RETENTION-004` | Retain and release compare expected and observed generations and publish exact successors only | unforgeable readiness and preflight proofs in `tests/retention_transition.rs` and `tests/retention_preflight.rs`; exact successor preparation and complete receipt evidence in `tests/retention_publication_preparation.rs` and `tests/retention_publication_execution.rs`; writer-locked initial filesystem publication in `filesystem_retention_storage_tests`; observed-head successor publication, exact predecessor binding, and absent-head refusal in `filesystem_retention_successor_tests`; the store's catalog head must name the closure's catalog generation and digest before any forward write in `filesystem_retention_catalog_tests`; a head whose predecessor disagrees with its manifest refuses in `filesystem_retention_current_tests`; a successor reopens and decodes the manifest-selected predecessor root and refuses an absent or changed one in `filesystem_retention_expectation_tests` | Implemented | | ||
| | `KEEP-RETENTION-005` | Closure derivation is deterministic, bounded, cycle-safe, fail-closed, and verifies complete blob reconstruction | exact accounting, reconstruction, adversarial-catalog, and exhaustive model laws in `tests/retention_closure.rs`; corrupt members refuse through the inherited segment-record admission laws and seeded `segment_format` fuzz target routed by `closure-corruption.md` | Implemented | | ||
| | `KEEP-RETENTION-006` | Publication follows the exact ordered durability protocol, including new namespace-directory admission and retention of fixed-stage evidence until head commit, and returns only after cleanup synchronization | typed vocabulary and blocking port in `tests/retention_publication_phase.rs` and `tests/retention_publication_storage.rs`; ordered execution, conditional namespace sync, and all 17 exact storage-fault boundaries in `tests/retention_publication_execution.rs`; production 17-phase forward filesystem execution, exclusive staging, byte-equal inode-substitution refusal, and retained-stage recovery refusal in `filesystem_retention_storage_tests`; orphan namespace directories count against the 4,096 ceiling and refuse a new namespace before any stage is written in `filesystem_retention_capacity_tests`; crash injection remains | In progress in #19 | | ||
| | `KEEP-RETENTION-007` | Restart resolves every fixed-stage crash prefix to one documented lawful state or typed ambiguity | recovery-required refusals before any mutation in `filesystem_retention_expectation_tests`: an absent head over populated pools, a non-initial head prepared against an absent head, an orphan directory for a namespace expected absent, and an absent directory for a namespace expected current; debug and release crash matrix remains | In progress in #19 | | ||
| | `KEEP-RETENTION-007` | Restart resolves every fixed-stage crash prefix to one documented lawful state or typed ambiguity | recovery-required refusals before any mutation in `filesystem_retention_expectation_tests`: an absent head over populated pools, a non-initial head prepared against an absent head, an orphan directory for a namespace expected absent, and an absent directory for a namespace expected current; debug and release crash matrix remains; replaced protocol directories, an absent or changed head-selected catalog, an over-full census, zero-generation pool names, and a stage retained by a failed write refuse in `filesystem_retention_*_tests` | In progress in #19 | | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The new evidence entry says that a changed head-selected catalog refuses in AGENTS.md reference: AGENTS.md:L123-L129 Useful? React with 👍 / 👎. |
||
| | `KEEP-RETENTION-008` | Readers double-collect catalog and retention heads and bind one complete catalog, manifest, and root-generation view under a `ReaderFence` | immutable snapshot and concurrency tests | Planned in #19 | | ||
| | `KEEP-RETENTION-009` | Exact already-committed retry is idempotent only while its successor remains current | byte-identical planning in `tests/retention_transition.rs`; authority-revalidated zero-mutation retry receipt in `tests/retention_publication_execution.rs`; exact already-committed filesystem retry with a byte-identical retention witness in `filesystem_retention_storage_tests`; superseded-candidate filesystem refusal with zero mutation in `filesystem_retention_successor_tests`; committed retry reopens the head-selected manifest entry and root pool bytes, refusing absent, changed, or corrupt evidence in `filesystem_retention_current_tests`; every refusal is a typed `RetentionCurrentStateRefusal` source, with superseded, committed-root-absent, committed-root-changed, and head-absent-with-artifacts pinned by downcast | Implemented | | ||
| | `KEEP-RETENTION-010` | Model operation sequences agree with a deterministic namespace-to-anchor-set map and never admit caller identity, paths, clocks, or application policy | model-based and source-architecture tests | Planned in #19 | | ||
|
|
@@ -30,7 +30,7 @@ case is not evidence. | |
| | --- | --- | --- | --- | | ||
| | `KEEP-MIGRATION-001` | Exact version-1 stores remain admitted until a durable migration artifact exists | compatibility fixtures | Planned in #19 | | ||
| | `KEEP-MIGRATION-002` | Format marker, intent, and receipt have complete fixed byte tables, named domains, bounds, checksums, deterministic store identity, and exact initial-state digests | exact admission in `tests/store_format_marker.rs`, `tests/store_migration_intent.rs`, and `tests/store_migration_receipt.rs`; canonical construction in `tests/store_migration_intent_encoding.rs` and `tests/store_migration_receipt_encoding.rs`; seeded `migration_format` fuzz target | Implemented | | ||
| | `KEEP-MIGRATION-003` | Migration revalidates version-1 head, catalog, pools, root identity, and writer authority before mutation | bounded canonical pool inventory in `tests/store_migration_inventory.rs`; writer-locked filesystem pool admission in `filesystem_inventory_*_tests`; exact authority observation and drift refusal in `filesystem_migration_authority_tests`; verification-first execution in `tests/store_migration_execution.rs`; fresh filesystem integration and post-publication drift refusal in `filesystem_migration_storage_tests` | Implemented | | ||
| | `KEEP-MIGRATION-003` | Migration revalidates version-1 head, catalog, pools, root identity, and writer authority before mutation | bounded canonical pool inventory in `tests/store_migration_inventory.rs`; writer-locked filesystem pool admission in `filesystem_inventory_*_tests`; exact authority observation and drift refusal in `filesystem_migration_authority_tests`; verification-first execution in `tests/store_migration_execution.rs`; fresh filesystem integration and post-publication drift refusal in `filesystem_migration_storage_tests`; a version-one store still holding a retained stage refuses before the intent is observed in `filesystem_migration_storage_tests` | Implemented | | ||
| | `KEEP-MIGRATION-004` | Every partial migration prefix continues idempotently under writer authority | state-machine and recovery tests | Planned in #19 | | ||
| | `KEEP-MIGRATION-005` | Unknown, out-of-order, substituted, corrupt, conflicting, or changed evidence is unrecoverable ambiguity | forward-execution stage preservation, byte-equal inode-substitution, out-of-order-prefix, and post-publication drift laws in `filesystem_migration_storage_tests`; unknown `retention` entries, non-digest namespace directories, and noncanonical pool names refuse before any retention stage is written in `filesystem_retention_namespace_tests`; restart corruption and mutation matrix remains | In progress in #19 | | ||
| | `KEEP-MIGRATION-006` | Migration never rewrites or deletes admitted version-1 immutable bytes | exact segment, catalog, and head before/after witness in `filesystem_migration_storage_tests`; restart-path evidence remains | In progress in #19 | | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When the candidate namespace changes after
verify_current, this promise is false:execute_rootwrites and synchronizesroot.nextbefore callingadmit_root_namespace(src/adapters/retention/publication_execution.rs:55-65). The regression atfilesystem_retention_attempt_tests.rs:78-102demonstrates exactly this ordering by creating the conflicting namespace after the stage write and then receivingNamespaceExpectationViolated; the retained stage subsequently requires recovery. Limit the claim to violations observed during preflight, or document that a later namespace refusal can leave durable stage evidence.AGENTS.md reference: AGENTS.md:L84-L89
Useful? React with 👍 / 👎.