From c81cd33060b8157b6aa99a6854992630d067fde6 Mon Sep 17 00:00:00 2001 From: callumalpass Date: Thu, 12 Nov 2026 06:58:45 +1100 Subject: [PATCH 1/7] Specify eight rules that tools had to guess (rc.6) slugify, link targets by path key, link and tag syntax, code in the body, frontmatter delimiters, the order_by value order and path tie-break, one write order in Chapter 12 with the create membership path, and the path_value_normalized warning. Fixtures and requirements for each; SN-103. --- 03-records-and-frontmatter.md | 26 ++- 04-configuration.md | 4 + 05-type-files.md | 3 +- 07-collection-semantics.md | 10 +- 08-links.md | 83 ++++++- 09-lifecycle.md | 56 +++-- 11-querying.md | 43 +++- 12-operations.md | 61 +++-- 14-conformance.md | 11 +- CHANGELOG.md | 61 +++++ notes/SN-103.md | 69 ++++++ tests/v0.3/cel/query-order.yaml | 275 +++++++++++++++++++++++ tests/v0.3/core/core-collection.yaml | 84 +++++++ tests/v0.3/core/core-write.yaml | 117 ++++++++++ tests/v0.3/core/links-and-discovery.yaml | 140 ++++++++++++ tests/v0.3/core/path-fields.yaml | 48 ++++ tests/v0.3/lifecycle/lifecycle.yaml | 71 ++++++ tests/v0.3/manifest.yaml | 10 + 18 files changed, 1112 insertions(+), 60 deletions(-) create mode 100644 notes/SN-103.md create mode 100644 tests/v0.3/cel/query-order.yaml diff --git a/03-records-and-frontmatter.md b/03-records-and-frontmatter.md index 2a7f371..a099bff 100644 --- a/03-records-and-frontmatter.md +++ b/03-records-and-frontmatter.md @@ -33,11 +33,27 @@ status: open Body text. ``` -If the first non-byte-order-mark bytes are not `---` followed by a line ending, -the file has no frontmatter and the full file is body text. - -Whitespace or a blank line before the opening delimiter means there is no -frontmatter. +The **opening delimiter** is `---` followed by a line ending (`\n` or +`\r\n`), as the first bytes of the file after an optional byte-order mark. +The **closing delimiter** is the first later line that is exactly `---`, +ended by a line ending or by the end of the file. The frontmatter source is +the text between the two delimiter lines, and the body is everything after +the closing delimiter's line ending, so a blank line that follows the +delimiter is part of the body. + +A file has no frontmatter, and the whole file is body text, when: + +- it does not begin with the opening delimiter; whitespace or a blank line + before `---`, and `---` followed by a space or any other character, are + not an opening delimiter +- no later line is a closing delimiter; an opening delimiter that is never + closed does not make the rest of the file frontmatter + +Only `---` closes frontmatter. The YAML document end marker `...`, a `---` +line with leading or trailing whitespace, and a longer run of dashes do not. +The first line that is exactly `---` closes it even when a YAML parser would +read that line as part of a block scalar, so every tool finds the same +frontmatter without parsing YAML. ## YAML Document Record Structure diff --git a/04-configuration.md b/04-configuration.md index 8c22791..d4980b0 100644 --- a/04-configuration.md +++ b/04-configuration.md @@ -100,6 +100,10 @@ exception is a uniqueness rule that explicitly opts into `enforce: write` single-record checks whose severity is always `warning`: they are reported at levels `warn` and `error` and never reject a write. +`path_value_normalized` (Chapter 07) is not a validation issue. It is a +warning in the result of the write that normalized the value, at every +validation level, and it is not reported when the record is read later. + Writes that do not go through an engine's write operations are never rejected. That includes edits made by other tools, files that appear through a file system or synchronization tool, and the results of the merge in Chapter 12A. diff --git a/05-type-files.md b/05-type-files.md index f5f81ed..93b95fb 100644 --- a/05-type-files.md +++ b/05-type-files.md @@ -81,7 +81,7 @@ and collection rules. Write operations use the same model around lifecycle processing: determine membership from the requested draft, run the applicable lifecycle assignments, then confirm membership once before final validation. The complete write order -is defined in Chapters 09 and 12. +is defined in Chapter 12. ## Required Frontmatter @@ -250,6 +250,7 @@ participates independently in each declaring type's uniqueness set. Write operations determine membership from the requested draft and target path before lifecycle runs. Explicit declarations use the same precedence as reads. +Chapter 12 defines which path a create uses when the request gives none. After lifecycle has run, the implementation MUST evaluate membership once more. If the final membership differs from the pre-lifecycle membership, the operation diff --git a/07-collection-semantics.md b/07-collection-semantics.md index 20e599e..267b7b9 100644 --- a/07-collection-semantics.md +++ b/07-collection-semantics.md @@ -487,8 +487,14 @@ path-stored field written `{field}` is placed in a path, it is normalized: 4. leading and trailing characters that are `.` or a space are removed. The normalized value replaces the value that was written: it is the record's -value and the value the operation result reports. Normalizing a normalized -value changes nothing. A value that is not a string, or that is empty after +value and the value the operation result reports. When normalization changes +a value that the request or a lifecycle action set, the operation result +carries a `warning` diagnostic with code `path_value_normalized`, the field, +`details.written` holding the value as it was set, and `details.stored` +holding the normalized value, so a caller that wrote `Plan: Q3/Q4` learns +that the record's title is `Plan Q3Q4` without comparing the two itself. The +diagnostic belongs to the operation, not to the record: reading the record +later does not report it. Normalizing a normalized value changes nothing. A value that is not a string, or that is empty after normalization, fails with `path_value_invalid` naming the field, and so does a value that gives the derived path a segment the portable path rules of Chapter 02 forbid, such as the Windows device name `CON` or a segment longer diff --git a/08-links.md b/08-links.md index bae639e..c3f4492 100644 --- a/08-links.md +++ b/08-links.md @@ -26,6 +26,39 @@ Parsed links expose: `format` is one of `wikilink`, `markdown`, or `path`. +## Link Syntax + +A **wikilink** is `[[`, a target, an optional `#anchor`, an optional `|alias`, +and `]]`, on one line. The alias is the text after the first `|`. In the text +before the alias, the anchor is the text after the first `#` and the target +is the text before it, with leading and trailing whitespace removed. A +wikilink with an empty target is not a link. The link ends at the first `]]`, +except that a `]` closing a `[` opened inside the link belongs to the link: +`[[Person [draft]]]` has the target `Person [draft]`. `![[...]]` is an embed. + +A **Markdown link** is `[text](destination)`, where `text` contains no `[` or +line break and `(` directly follows `]`. `![text](destination)` is an embed. +The destination has one of two forms: + +- `<...>`: the destination is the text between the angle brackets exactly as + written. Only whitespace or a title that begins with `"` may follow it + before `)`. +- otherwise the destination is the text up to the first space or `)`. Text + between a space and `)`, such as a `"title"`, is ignored, and each `%20` in + the destination is read as a space. No other percent-encoding is decoded, + so a destination that contains a space or `)` is written with `%20` or in + angle brackets. + +The anchor is the text after the first `#` of the destination, and the target +is the text before it. A Markdown link is not a record link, and is ignored, +when its destination is empty, begins with `#`, or begins with a URI scheme: +an ASCII letter, then ASCII letters, digits, `+`, `-`, or `.`, then `:`, as in +`https:` or `mailto:`. + +A **bare path** is a complete field value that is neither of the above, is +not empty, contains no `[`, `]`, or line break, and does not begin with a URI +scheme. Its anchor is the text after the first `#`. + ## Resolution Resolution is performed against the collection root and the containing file. @@ -40,6 +73,10 @@ General rules: resolve from the collection root. - Simple wikilinks without path separators resolve by filename, matching the target against record filenames with or without their record extension. +- A path that names no record as written, and whose last segment does not + end in a record extension, is tried with each extension of + `settings.record_extensions` appended, in the configured order. The first + path that names a record is the target. - When `settings.id_field` is configured, a simple wikilink first tries ID-based resolution against that field and falls back to filename resolution only when no record has that ID. When `settings.id_field` is @@ -49,7 +86,13 @@ A record has an ID when its persisted `id_field` value is a non-empty string. ID-based resolution compares the wikilink target with record IDs exactly, without case folding or normalization. -After normalization, a link that escapes the collection root is invalid. +Targets are compared with record paths and filenames by path key (Chapter +02), so `[[Alice]]` and `[[people/Alice]]` resolve to `people/alice.md`. Two +records whose paths have the same path key are already a `path_collision`. +An ID is compared exactly, as above. + +After `.` and `..` segments are resolved, a link that escapes the collection +root is invalid. ## Ambiguity @@ -139,18 +182,44 @@ backlinks. Backlinks are derived from the current collection state. After a successful write, subsequent reads and queries observe backlinks that reflect it. +## Code In The Body + Links and tags inside fenced code blocks and inline code MUST be ignored by -body extraction. +body extraction. Body extraction reads the body one line at a time and +recognizes code as follows, which is a subset of CommonMark: + +- A **fenced code block** opens on a line that is indented by fewer than + four spaces and whose first other characters are a run of at least three + backticks or at least three tildes. A backtick run opens a block only when + the rest of its line contains no backtick. The block closes on a line that + is indented by fewer than four spaces and begins with a run of the same + character at least as long as the opening run, followed only by + whitespace. A block that is never closed runs to the end of the body. The + opening and closing lines are part of the block. +- **Inline code** begins, on a line outside a fenced code block, at a run of + backticks, and ends after the next place on the same line where that many + backticks occur. A run with no such place is ordinary text. +- A backslash makes the character after it ordinary text, so `\[[x]]` is not + a link and a backtick after a backslash opens no inline code. + +Indented code blocks are not recognized: a link or tag on an indented line is +extracted. ## Tags -`file.tags` includes: +`file.tags` lists, in this order and without de-duplication: -- frontmatter tags from `tags` when the field is string or list of strings -- inline body tags beginning with `#` +- the frontmatter `tags` value when it is a string, or its string items when + it is a list, each with any leading `#` characters removed +- the inline tags of the body, in order of appearance -Inline tags must begin at the start of a line or after whitespace. URL -fragments MUST NOT be treated as tags. +An inline tag is a `#` that is the first character of its line or follows a +whitespace character (Unicode `White_Space`), followed by the longest run of tag characters. A tag +character is a letter or digit (Unicode `Alphabetic`, or general category +`Nd`, `Nl`, or `No`), `_`, `-`, or `/`. The tag is the run, without the `#`. +A run that is empty, or that consists only of ASCII digits, is not a tag, so +`# Heading`, `#123`, and the fragment of `https://example.com/a#section` are +not tags, while `#2026-plan` and `#project/alpha` are. `file.hasTag("project")` uses complete tag-segment prefixes. It matches `#project` and `#project/alpha`; its result for `#projection` is false. diff --git a/09-lifecycle.md b/09-lifecycle.md index a5cf0ee..05ad39a 100644 --- a/09-lifecycle.md +++ b/09-lifecycle.md @@ -73,10 +73,25 @@ An implementation MAY offer a sequence provider under an `x-*` extension; such a provider is not portable, and its allocation and coordination are implementation behavior. -`slugify` lowercases the value, transliterates it to ASCII where a -transliteration exists, replaces each run of other characters with `-`, and -trims leading and trailing `-`. A missing, null, or non-string source value -produces null. `copy` of a missing field removes the target key. +`slugify` builds its slug from a string value in four steps: + +1. Decompose the value to Unicode Normalization Form D and remove every + combining mark (general category `Mn`, `Mc`, or `Me`), using the Unicode + version of Chapter 02. +2. Replace each ASCII letter `A` to `Z` with its lowercase letter. +3. Replace each run of characters other than `a` to `z` and `0` to `9` with + one `-`. +4. Remove a leading and a trailing `-`. + +So `Café — Crème Brûlée!` gives `cafe-creme-brulee`. Only canonical +decomposition removes an accent: a character that does not decompose to an +ASCII letter is a separator like any other, so `Straße` gives `stra-e`, +`Ærø` gives `r`, and a value without any ASCII letter or digit, such as +`日本語`, gives the empty string. There is no other transliteration, so every +tool produces the same slug. + +A missing, null, or non-string source value produces null. `copy` of a +missing field removes the target key. ## Guards @@ -112,24 +127,21 @@ value instead of conflicting. A merge does not run lifecycle: lifecycle values in a merged record come from the two sides through their merge strategies (Chapter 12A). -## Validation Order - -For create and update: - -1. parse input -2. build a draft frontmatter object -3. determine and freeze type membership from the draft and target path -4. apply the lifecycle actions of every matched type -5. re-evaluate membership and fail with `type_membership_changed` if it differs -6. validate JSON Schema -7. run collection validators -8. write the file - -Lifecycle MUST run before final validation so generated IDs and timestamps can -satisfy required schema fields. It also runs before a path is derived from -path-stored fields (Chapter 07): the draft holds the persisted values, -including path-stored values, and a lifecycle action that sets a path-stored -field determines the record's path. Chapters 05 and 12 define membership freezing. +## Place In The Write Order + +Chapter 12 defines the complete order of a create and of an update. Within +it, lifecycle: + +- runs once per operation, after type membership is frozen from the draft, + for exactly the frozen types +- runs before JSON Schema and collection validation, so generated IDs and + timestamps can satisfy required schema fields +- runs before a path is derived from path-stored fields (Chapter 07): the + draft holds the persisted values, including path-stored values, and a + lifecycle action that sets a path-stored field determines the record's + path +- is followed by a second evaluation of membership, which fails the + operation with `type_membership_changed` when lifecycle changed it Read defaults MUST NOT run as lifecycle policy unless a write operation explicitly asks to materialize them. diff --git a/11-querying.md b/11-querying.md index a8b99d0..0481b15 100644 --- a/11-querying.md +++ b/11-querying.md @@ -165,10 +165,43 @@ order_by: direction: desc ``` -Null values sort last in ascending order and first in descending order. - -When all ordering fields compare equal, tools MUST tie-break by ascending -`file.path` for deterministic results. +Every pair of values has a defined order, so a result order never depends on +the tool. In ascending order, values sort first by kind: + +| Rank | Kind | Order within the kind | +| --- | --- | --- | +| 1 | boolean | `false` before `true` | +| 2 | number | by exact numeric value; integers and non-integers share one order, so `2` sorts before `2.5` and equals `2.0` | +| 3 | date or date-time | by instant; a date is the start of that day in UTC | +| 4 | string | by Unicode code point, without case folding or normalization | +| 5 | list | by number of items | +| 6 | map | by number of keys | +| 7 | null or missing | all equal | + +A string is of the date or date-time kind only when the tool knows it is one: + +- the value of a field of a typed record, when every type the record + matches declares the same `format: date` or the same `format: date-time` + for that field, and the value is valid for that format and, for a + date-time, has an offset +- a projection or selection value that is a CEL timestamp + +Every other string is of the string kind, however it is written: a string +that merely looks like a date is not guessed to be one, and an invalid value +in a `format: date` field sorts as a string, after every valid date. + +Two lists with the same number of items, and two maps with the same number +of keys, compare equal whatever they contain. A query that needs an order +over such values orders by a projection. + +`direction: desc` reverses this whole order for its term, so null and missing +values sort last in ascending order and first in descending order. + +When all ordering terms compare equal, tools MUST tie-break by ascending +`file.path` in Unicode code-point order, whatever the terms' directions. Paths +are unique, so the order of a result set is total, and `limit` and `offset` +always select the same records from the same collection state. A tool that +pages with its own continuation tokens MUST return the pages of this order. ## Grouping And Summaries @@ -183,7 +216,7 @@ group_by: Each distinct ordered tuple of grouping values creates one group. Null and missing values form the same null group. Group tuple ordering applies the same -direction and null rules as `order_by`. Results within a group retain query +value order and direction as `order_by`. Results within a group retain query order. Queries MAY define custom summary functions and apply built-in or custom diff --git a/12-operations.md b/12-operations.md index bcdc671..456a1ce 100644 --- a/12-operations.md +++ b/12-operations.md @@ -49,19 +49,52 @@ Create builds a new record. Pipeline: -1. determine target type or types from explicit input or path/match policy -2. build draft frontmatter -3. freeze pre-lifecycle type membership -4. apply lifecycle `on_create` -5. verify type membership did not change as a lifecycle side effect -6. validate JSON Schema -7. run collection validators, applying the validation tiers of Chapter 04 -8. choose or validate path policy; the final path's extension fixes the - record's format (Chapter 03), and a body is rejected when that format is a - YAML document -9. write Markdown file -10. update derived indexes -11. emit watch/runtime events after state is consistent +1. select the target type or types from explicit input, contract selection, + or path and match policy +2. build the draft frontmatter from the request, with the declaration of a + selected type (below); with an explicit `path`, add the path-stored + values decoded from it +3. choose the membership path and freeze type membership (below) +4. apply lifecycle `on_create` of the frozen types +5. when there is a membership path, evaluate membership again for the draft + after lifecycle and fail with `type_membership_changed` if it differs +6. normalize the path-stored values that the request or lifecycle set + (Chapter 07), and derive the path when the request gave none +7. check the final path: path safety, the record extension, collisions, and + that every frozen type matches the record at it, failing with + `type_membership_changed` otherwise; the path's extension fixes the + record's format (Chapter 03), and a body is rejected when that format is + a YAML document +8. validate JSON Schema for every type that matches at the final path +9. run collection validators, applying the validation tiers of Chapter 04 +10. write the file +11. update derived indexes +12. emit watch/runtime events after state is consistent + +This list and the update pipeline below are the write order. Chapters 05 and +09 describe type membership and lifecycle within it. + +The **membership path** is the path that `match` rules read in step 3, +before lifecycle has run and before a path can always be derived: + +- An explicit `path` is the membership path. +- Otherwise, when no selected or matched type has path-stored fields and the + path pattern can already be derived from the draft, the derived path is + the membership path. The pattern is that of the selected type, or else of + the first type in canonical order that the draft matches without a path. + This path is provisional: it is used only to evaluate membership, it is + not checked for collisions, and the final path is derived again in step 6, + after lifecycle. +- Otherwise there is no membership path. Membership is evaluated from the + draft's frontmatter with an empty path, and step 5 is skipped. A pattern + with path-stored fields never gives a provisional path, because its values + are normalized only after lifecycle. + +The frozen membership is the types that match the draft at the membership +path, together with the selected type. Lifecycle runs for exactly those +types. A type that matches only through the final path, when there was no +membership path, joins the record for validation in steps 8 and 9, and its +lifecycle does not run. An explicit `path` whose path key (Chapter 02) equals that of an existing record fails with `path_conflict`. A path derived from `collection.path.pattern` @@ -70,7 +103,7 @@ free suffixed path from Chapter 02, and the result reports the final path. When a selected type has path-stored fields (Chapter 07): -- Without an explicit `path`, the path is derived in step 8 from the values +- Without an explicit `path`, the path is derived in step 6 from the values after lifecycle `on_create`, so a lifecycle action can supply a path-stored value. Each path-stored value is normalized first. A derived path whose path key equals that of an existing record fails with diff --git a/14-conformance.md b/14-conformance.md index 70e0a3b..483824c 100644 --- a/14-conformance.md +++ b/14-conformance.md @@ -152,7 +152,8 @@ The v0.3 core codes include `invalid_request`, `duplicate_batch_path`, `projection_shadowed`, `link_not_found`, `link_target_type_mismatch`, `ambiguous_link`, `duplicate_value`, `path_conflict`, `path_collision`, `type_conflict`, `type_membership_changed`, `path_value_missing`, -`path_value_invalid`, `path_value_inconsistent`, `path_field_in_frontmatter`, +`path_value_invalid`, `path_value_normalized`, `path_value_inconsistent`, +`path_field_in_frontmatter`, `unknown_path_key`, `invalid_type`, `nondeterministic_match`, `invalid_pattern`, `schema_ref_forbidden`, `schema_ref_unresolved`, `schema_ref_cycle`, `format_invalid`, @@ -287,7 +288,8 @@ Query implementations MUST: - evaluate `where` filters against the effective query context - evaluate requested CEL projections - support OR-based type filtering -- support deterministic ordering and pagination +- order values with the comparator of Chapter 11 and tie-break by ascending + `file.path`, so ordering and pagination are deterministic - support deterministic grouping and built-in and custom summaries - return total-count and has-more metadata - return context, grouping, and summary metadata when requested @@ -390,8 +392,9 @@ Core Write implementations MUST: - reject an explicit path that collides under path equivalence with `path_conflict`, give a colliding derived path the first free suffix, and reject invalid path-pattern values with `path_value_invalid` -- never persist path-stored fields to frontmatter, normalize their values, - derive paths from them after lifecycle, reject a colliding derived path +- never persist path-stored fields to frontmatter, normalize their values + and report `path_value_normalized` when that changes one, derive paths + from them after lifecycle, reject a colliding derived path with `path_conflict`, and move a record atomically when an update changes a path-stored field - enforce `enforce: write` uniqueness rules in a single write order diff --git a/CHANGELOG.md b/CHANGELOG.md index 8c7b914..ee87d00 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -54,6 +54,67 @@ values of [SN-056](./notes/SN-056.md); see [SN-102](./notes/SN-102.md). - `path_value_missing` and `path_value_invalid` on a write are listed in the request and safety tier (Chapter 04). +### Rules that tools previously had to guess + +Eight places where two conforming tools could disagree. See +[SN-103](./notes/SN-103.md). + +#### Added + +- `slugify` is an algorithm (Chapter 09): decompose to NFD and drop + combining marks, lowercase ASCII letters, replace each run of other + characters with one `-`, and trim `-`. There is no transliteration, so + `Straße` gives `stra-e` and `日本語` gives the empty string. +- Link targets are compared with record paths and filenames by path key, so + `[[Alice]]` resolves to `people/alice.md`. IDs are still compared exactly. + A path without a record extension is tried with each configured extension + in order (Chapter 08). +- Link syntax (Chapter 08): the wikilink grammar, including a `]` that + closes a `[` opened inside the link; Markdown link destinations, in angle + brackets as written or up to the first space or `)` with `%20` read as a + space and no other decoding; destinations that are empty, begin with `#`, + or begin with a URI scheme are not record links. +- Code in the body (Chapter 08): which lines open and close a fenced code + block, where inline code ends, and that a backslash makes the next + character ordinary text. Link and tag extraction use the same rule. +- Inline tag grammar (Chapter 08): `#` at the start of a line or after + whitespace, then letters, digits, `_`, `-`, or `/`. A run of ASCII digits + alone is not a tag. A leading `#` in a frontmatter `tags` value is + removed. +- Frontmatter delimiters (Chapter 03): the opening delimiter is `---` and a + line ending at the start of the file; the closing delimiter is the first + later line that is exactly `---`. `...`, `--- `, and a longer dash run do + not close frontmatter, and a file whose frontmatter is never closed has + none. +- `order_by` value order (Chapter 11): booleans, numbers, dates and + date-times, strings, lists, maps, then null and missing. A string is a + date or date-time only where every matched type declares that format for + the field. Lists and maps compare by length. +- `path_value_normalized` (Chapters 07 and 14): a warning in the result of a + write when normalization changed a path-stored value, with the written and + stored values. +- Fixtures marked `since: 0.3.0-rc.6` in `core/links-and-discovery.yaml`, + `core/core-collection.yaml`, `core/core-write.yaml`, + `core/path-fields.yaml`, `lifecycle/lifecycle.yaml`, and the new + `cel/query-order.yaml`. New requirements + `core_read.frontmatter_delimiters`, `links.path_key_matching`, + `links.markdown_destinations`, `links.body_code_and_tags`, + `cel_query.value_order`, `cel_query.path_tiebreak`, + `core_write.path_value_normalized`, `core_write.create_membership_path`, + and `lifecycle.slugify`. + +#### Changed + +- Chapter 12 holds the one write order. The create pipeline names the + membership path: the explicit path, or the path the draft derives before + lifecycle when the pattern has no path-stored field, or none. Types are + frozen at that path, lifecycle runs for the frozen types, and every frozen + type must still match at the final path. Chapters 05 and 09 point to + Chapter 12 instead of repeating the order. +- The final `order_by` tie-break is ascending `file.path` in code-point + order whatever the directions of the terms, and applies to tools that page + with their own continuation tokens. + ## 0.3.0-rc.5 (draft, untagged) The fifth release candidate specifies the data semantics that let several diff --git a/notes/SN-103.md b/notes/SN-103.md new file mode 100644 index 0000000..c924bf2 --- /dev/null +++ b/notes/SN-103.md @@ -0,0 +1,69 @@ +--- +id: SN-103 +title: "Rules that two conforming tools could read differently" +sections: + - "§3 Records and Frontmatter" + - "§8 Links" + - "§9 Lifecycle" + - "§11 Querying" + - "§12 Operations" +status: resolved +kind: ambiguity +severity: medium +--- + +**Sections:** §3 Records and Frontmatter, §8 Links, §9 Lifecycle, §11 +Querying, §12 Operations + +**Issue:** A review of rc.5 and the rc.6 draft against the mdbase-next Core +found eight rules that the text left to the implementation. Each one changes +stored data or a visible result, so two conforming tools could disagree on +the same collection: + +1. `slugify` was "a slug of another field's value" with no algorithm. +2. Simple wikilinks matched filenames, but the text did not say whether + `[[Alice]]` matches `alice.md`. +3. The closing frontmatter delimiter was not defined, so `...`, `--- `, and + an unclosed block were open questions. +4. Markdown link destinations had no grammar: percent-encoding, angle + brackets, titles, and external URLs. +5. Inline tags had no grammar: `#123`, `# Heading`, URL fragments, and which + characters end a tag. +6. The write order appeared in Chapters 05, 09, and 12 with different + steps, and none said which path freezes type membership on a create + whose path is derived. +7. `order_by` said how to compare values of one kind but not values of + different kinds, and Core had moved the final tie-break from the path to + the record ID. +8. Normalizing a path-stored value changes what the user typed, such as + `Plan: Q3/Q4` to `Plan Q3Q4`, and nothing told the caller. + +**Resolution (0.3.0-rc.6):** + +1. Chapter 09 gives the four steps of `slugify`. It has no transliteration + table, because a table is a second thing every tool would have to copy + exactly. +2. Chapter 08 compares targets with paths and filenames by path key. IDs + stay exact. +3. Chapter 03 defines both delimiters by their bytes. Only a line that is + exactly `---` closes frontmatter, so a tool finds the end of frontmatter + without parsing YAML. +4. Chapter 08 defines the two destination forms and decodes only `%20`. +5. Chapter 08 defines the tag grammar and one rule for code in the body, + shared by link and tag extraction. It is a stated subset of CommonMark, + so a tool needs no Markdown parser. +6. Chapter 12 holds the only copy of the write order and defines the + membership path of a create. +7. Chapter 11 adopts Core's order of kinds and keeps `file.path` as the + final tie-break. A path is in every result row and a record ID is not, + so a caller can check and continue a path-ordered page without engine + state. Core returns to path ties. +8. Chapter 07 adds the `path_value_normalized` warning to the result of the + write. + +**Conformance impact:** fixtures marked `since: 0.3.0-rc.6` under requirements +`lifecycle.slugify`, `links.path_key_matching`, +`links.markdown_destinations`, `links.body_code_and_tags`, +`core_read.frontmatter_delimiters`, `core_write.create_membership_path`, +`cel_query.value_order`, `cel_query.path_tiebreak`, and +`core_write.path_value_normalized`. diff --git a/tests/v0.3/cel/query-order.yaml b/tests/v0.3/cel/query-order.yaml new file mode 100644 index 0000000..40ace22 --- /dev/null +++ b/tests/v0.3/cel/query-order.yaml @@ -0,0 +1,275 @@ +name: "v0.3 query ordering conformance" +spec_version: "0.3.0" +fixture_set: cel +category: query +spec_ref: "v0.3/11" + +groups: + - name: "value order across kinds" + setup: + config: | + spec_version: "0.3.0" + files: + items/bool-false.md: | + --- + v: false + --- + items/bool-true.md: | + --- + v: true + --- + items/num-neg.md: | + --- + v: -1 + --- + items/num-2.md: | + --- + v: 2 + --- + items/num-2-0.md: | + --- + v: 2.0 + --- + items/num-2-5.md: | + --- + v: 2.5 + --- + items/str-10.md: | + --- + v: "10" + --- + items/str-B.md: | + --- + v: B + --- + items/str-a.md: | + --- + v: a + --- + items/list-1.md: | + --- + v: [z] + --- + items/list-2.md: | + --- + v: [a, b] + --- + items/map-1.md: | + --- + v: { k: 1 } + --- + items/null.md: | + --- + v: null + --- + items/missing.md: | + --- + other: 1 + --- + tests: + - name: "ascending order sorts by kind, then within the kind, with null and missing last" + id: query.order_kinds_ascending + since: 0.3.0-rc.6 + covers: [cel_query.value_order, cel_query.path_tiebreak] + operation: query + input: + order_by: + - field: v + direction: asc + expect: + valid: true + results: + - path: "items/bool-false.md" + - path: "items/bool-true.md" + - path: "items/num-neg.md" + - path: "items/num-2-0.md" + - path: "items/num-2.md" + - path: "items/num-2-5.md" + - path: "items/str-10.md" + - path: "items/str-B.md" + - path: "items/str-a.md" + - path: "items/list-1.md" + - path: "items/list-2.md" + - path: "items/map-1.md" + - path: "items/missing.md" + - path: "items/null.md" + + - name: "descending order reverses the value order and keeps ties in ascending path order" + id: query.order_kinds_descending + since: 0.3.0-rc.6 + covers: [cel_query.value_order, cel_query.path_tiebreak] + operation: query + input: + order_by: + - field: v + direction: desc + expect: + valid: true + results: + - path: "items/missing.md" + - path: "items/null.md" + - path: "items/map-1.md" + - path: "items/list-2.md" + - path: "items/list-1.md" + - path: "items/str-a.md" + - path: "items/str-B.md" + - path: "items/str-10.md" + - path: "items/num-2-5.md" + - path: "items/num-2-0.md" + - path: "items/num-2.md" + - path: "items/num-neg.md" + - path: "items/bool-true.md" + - path: "items/bool-false.md" + + - name: "schema-typed dates and date-times" + setup: + config: | + spec_version: "0.3.0" + types: + event.md: | + --- + kind: mdbase.type + name: event + version: 1 + match: + path_glob: "events/**/*.md" + schema: + dialect: json-schema-2020-12 + value: + type: object + properties: + day: { type: string, format: date } + at: { type: string, format: date-time } + --- + files: + events/a.md: | + --- + day: "2026-01-10" + at: "2026-01-01T10:00:00+10:00" + --- + events/b.md: | + --- + day: "2025-12-31" + at: "2026-01-01T01:00:00Z" + --- + events/c.md: | + --- + day: "0 someday" + at: "2026-01-01T00:30:00Z" + --- + loose/a.md: | + --- + at: "2026-01-01T10:00:00+10:00" + --- + loose/b.md: | + --- + at: "2026-01-01T01:00:00Z" + --- + tests: + - name: "date-times of a format: date-time field sort by instant, not by text" + id: query.order_date_time_instants + since: 0.3.0-rc.6 + covers: [cel_query.value_order] + operation: query + input: + types: [event] + order_by: + - field: at + expect: + valid: true + results: + - path: "events/a.md" + - path: "events/c.md" + - path: "events/b.md" + + - name: "an invalid value in a format: date field sorts as a string, after every date" + id: query.order_invalid_date_is_string + since: 0.3.0-rc.6 + covers: [cel_query.value_order] + operation: query + input: + types: [event] + order_by: + - field: day + expect: + valid: true + results: + - path: "events/b.md" + - path: "events/a.md" + - path: "events/c.md" + + - name: "a string that looks like a date-time is not guessed to be one without a schema" + id: query.order_untyped_strings + since: 0.3.0-rc.6 + covers: [cel_query.value_order] + operation: query + input: + where: 'file.inFolder("loose")' + order_by: + - field: at + expect: + valid: true + results: + - path: "loose/b.md" + - path: "loose/a.md" + + - name: "path tie-break" + setup: + config: | + spec_version: "0.3.0" + files: + tasks/b.md: | + --- + status: open + rank: 1 + --- + tasks/a.md: | + --- + status: open + rank: 2 + --- + tasks/C.md: | + --- + status: open + rank: 2 + --- + tasks/d.md: | + --- + status: done + rank: 2 + --- + tests: + - name: "equal terms fall back to ascending file.path in code-point order" + id: query.order_path_tiebreak_ascending + since: 0.3.0-rc.6 + covers: [cel_query.path_tiebreak] + operation: query + input: + where: 'status == "open"' + order_by: + - field: status + expect: + valid: true + results: + - path: "tasks/C.md" + - path: "tasks/a.md" + - path: "tasks/b.md" + + - name: "the tie-break stays ascending when every term is descending" + id: query.order_path_tiebreak_descending + since: 0.3.0-rc.6 + covers: [cel_query.path_tiebreak] + operation: query + input: + order_by: + - field: rank + direction: desc + - field: status + direction: desc + expect: + valid: true + results: + - path: "tasks/C.md" + - path: "tasks/a.md" + - path: "tasks/d.md" + - path: "tasks/b.md" diff --git a/tests/v0.3/core/core-collection.yaml b/tests/v0.3/core/core-collection.yaml index 0cff155..7bac846 100644 --- a/tests/v0.3/core/core-collection.yaml +++ b/tests/v0.3/core/core-collection.yaml @@ -716,3 +716,87 @@ groups: valid: false error: code: path_traversal + + - name: "frontmatter delimiters" + setup: + config: | + spec_version: "0.3.0" + files: + notes/closed-at-eof.md: "---\ntitle: Eof\n---" + notes/blank-after.md: "---\ntitle: Blank\n---\n\nBody\n" + notes/unclosed.md: "---\ntitle: Unclosed\n\nBody\n" + notes/document-end.md: "---\ntitle: Dots\n...\nBody\n" + notes/opening-space.md: "--- \ntitle: Space\n---\nBody\n" + notes/blank-first.md: "\n---\ntitle: Late\n---\nBody\n" + tests: + - name: "a closing delimiter may end the file" + id: core_read.frontmatter_closed_at_eof + since: 0.3.0-rc.6 + covers: [core_read.frontmatter_delimiters] + operation: read + input: + path: "notes/closed-at-eof.md" + expect: + valid: true + frontmatter: { title: Eof } + body: "" + + - name: "the body starts after the closing delimiter's line ending" + id: core_read.frontmatter_body_start + since: 0.3.0-rc.6 + covers: [core_read.frontmatter_delimiters] + operation: read + input: + path: "notes/blank-after.md" + expect: + valid: true + frontmatter: { title: Blank } + body: "\nBody\n" + + - name: "an opening delimiter that is never closed means no frontmatter" + id: core_read.frontmatter_unclosed + since: 0.3.0-rc.6 + covers: [core_read.frontmatter_delimiters] + operation: read + input: + path: "notes/unclosed.md" + expect: + valid: true + frontmatter: {} + body: "---\ntitle: Unclosed\n\nBody\n" + + - name: "the YAML document end marker does not close frontmatter" + id: core_read.frontmatter_document_end + since: 0.3.0-rc.6 + covers: [core_read.frontmatter_delimiters] + operation: read + input: + path: "notes/document-end.md" + expect: + valid: true + frontmatter: {} + body: "---\ntitle: Dots\n...\nBody\n" + + - name: "an opening delimiter followed by a space is not a delimiter" + id: core_read.frontmatter_opening_space + since: 0.3.0-rc.6 + covers: [core_read.frontmatter_delimiters] + operation: read + input: + path: "notes/opening-space.md" + expect: + valid: true + frontmatter: {} + body: "--- \ntitle: Space\n---\nBody\n" + + - name: "a blank line before the opening delimiter means no frontmatter" + id: core_read.frontmatter_blank_first + since: 0.3.0-rc.6 + covers: [core_read.frontmatter_delimiters] + operation: read + input: + path: "notes/blank-first.md" + expect: + valid: true + frontmatter: {} + body: "\n---\ntitle: Late\n---\nBody\n" diff --git a/tests/v0.3/core/core-write.yaml b/tests/v0.3/core/core-write.yaml index c363956..316f957 100644 --- a/tests/v0.3/core/core-write.yaml +++ b/tests/v0.3/core/core-write.yaml @@ -472,3 +472,120 @@ groups: label: managed slug: caller-label stamp: second + + - name: "create membership path" + setup: + config: | + spec_version: "0.3.0" + settings: + explicit_type_keys: [] + types: + note.md: | + --- + kind: mdbase.type + name: note + version: 1 + match: + path_glob: "notes/**/*.md" + schema: + dialect: json-schema-2020-12 + value: + type: object + properties: + title: { type: string } + collection: + path: + pattern: "notes/{title}.md" + --- + stamped.md: | + --- + kind: mdbase.type + name: stamped + version: 1 + match: + path_glob: "notes/**/*.md" + schema: + dialect: json-schema-2020-12 + value: + type: object + properties: + stamp: { type: string } + lifecycle: + on_create: + set: + stamp: { literal: applied } + --- + generated.md: | + --- + kind: mdbase.type + name: generated + version: 1 + match: + path_glob: "gen/**/*.md" + schema: + dialect: json-schema-2020-12 + value: + type: object + properties: + title: { type: string } + slug: { type: string } + collection: + path: + pattern: "gen/{slug}.md" + lifecycle: + on_create: + set: + slug: { slugify: title } + --- + audited.md: | + --- + kind: mdbase.type + name: audited + version: 1 + match: + path_glob: "gen/**/*.md" + schema: + dialect: json-schema-2020-12 + value: + type: object + properties: + audited: { type: string } + lifecycle: + on_create: + set: + audited: { literal: applied } + --- + files: {} + tests: + - name: "a path the draft already derives freezes membership, so a path-matched type's lifecycle runs" + id: core_write.create_provisional_path + since: 0.3.0-rc.6 + covers: [core_write.create_membership_path] + operation: create + input: + type: note + frontmatter: + title: Plan + expect: + valid: true + path: "notes/Plan.md" + types: [note, stamped] + frontmatter_contains: + stamp: applied + + - name: "without a membership path only the selected type's lifecycle runs, and a path-matched type joins for validation" + id: core_write.create_no_membership_path + since: 0.3.0-rc.6 + covers: [core_write.create_membership_path] + operation: create + input: + type: generated + frontmatter: + title: Hello World + expect: + valid: true + path: "gen/hello-world.md" + types: [audited, generated] + frontmatter_contains: + slug: hello-world + frontmatter_not_contains: [audited] diff --git a/tests/v0.3/core/links-and-discovery.yaml b/tests/v0.3/core/links-and-discovery.yaml index 2aa07bf..118ff6c 100644 --- a/tests/v0.3/core/links-and-discovery.yaml +++ b/tests/v0.3/core/links-and-discovery.yaml @@ -226,3 +226,143 @@ groups: - code: ambiguous_link field: parent details: { candidates: ["tasks/a.md", "tasks/b.md"] } + + - name: "targets compare by path key" + setup: + config: | + spec_version: "0.3.0" + files: + people/alice.md: | + --- + title: Alice + --- + notes/ref.md: | + --- + simple: "[[Alice]]" + rooted: "[[People/ALICE]]" + with_ext: "[[people/Alice.MD]]" + missing: "[[Alicia]]" + --- + tests: + - name: "a simple wikilink matches a filename that differs only in case" + id: links.filename_case_insensitive + since: 0.3.0-rc.6 + covers: [links.path_key_matching] + operation: evaluate_cel + input: + path: "notes/ref.md" + expression: 'link(simple).asFile().file.path == "people/alice.md"' + expect: + valid: true + value: true + + - name: "a path target matches a path that differs only in case, with or without the extension" + id: links.path_case_insensitive + since: 0.3.0-rc.6 + covers: [links.path_key_matching] + operation: evaluate_cel + input: + path: "notes/ref.md" + expression: 'link(rooted).asFile().file.path == "people/alice.md" && link(with_ext).asFile().file.path == "people/alice.md" && link(missing).asFile() == null' + expect: + valid: true + value: true + + - name: "Markdown link destinations" + setup: + config: | + spec_version: "0.3.0" + files: + people/alice smith.md: | + --- + title: Alice Smith + --- + notes/md.md: | + --- + title: Markdown links + --- + [encoded](../people/alice%20smith.md) + [angle](<../people/alice smith.md>) + [titled](../people/alice%20smith.md "Alice") + [web](https://example.com/people/alice.md) + [mail](mailto:alice@example.com) + [fragment](#section) + [anchored](../people/alice%20smith.md#bio) + tests: + - name: "%20 is a space, angle brackets are literal, titles are dropped, and URIs and fragments are not record links" + id: links.markdown_destinations + since: 0.3.0-rc.6 + covers: [links.markdown_destinations] + operation: evaluate_cel + input: + path: "notes/md.md" + expression: 'file.links == ["../people/alice smith.md", "../people/alice smith.md", "../people/alice smith.md", "../people/alice smith.md"]' + expect: + valid: true + value: true + + - name: "each destination resolves to the record" + id: links.markdown_destinations_resolve + since: 0.3.0-rc.6 + covers: [links.markdown_destinations] + operation: evaluate_cel + input: + path: "people/alice smith.md" + expression: 'file.backlinks.size() == 1' + expect: + valid: true + value: true + + - name: "code and inline tags in the body" + setup: + config: | + spec_version: "0.3.0" + files: + notes/real.md: | + --- + title: Real + --- + notes/tags.md: | + --- + tags: ["#fm", plain] + --- + # Heading + #first tag, #project/alpha and x#nope + See https://example.com/a#fragment and issue #123 and #2026-plan. + `#code` and ``a ` #code2`` then #after + \[[escaped]] [[real]] + + ```md + #fenced [[fenced]] + ``` + + ~~~~ + ``` + #still-fenced [[still-fenced]] + ~~~~ + #indented + #café_日本 + tests: + - name: "inline tags follow the tag grammar and skip code" + id: links.inline_tags + since: 0.3.0-rc.6 + covers: [links.body_code_and_tags] + operation: evaluate_cel + input: + path: "notes/tags.md" + expression: 'file.tags == ["fm", "plain", "first", "project/alpha", "2026-plan", "after", "indented", "café_日本"]' + expect: + valid: true + value: true + + - name: "links in fenced blocks, inline code, and after a backslash are ignored" + id: links.body_code_links + since: 0.3.0-rc.6 + covers: [links.body_code_and_tags] + operation: evaluate_cel + input: + path: "notes/tags.md" + expression: 'file.links == ["[[real]]"] && file.hasTag("project") && !file.hasTag("proj")' + expect: + valid: true + value: true diff --git a/tests/v0.3/core/path-fields.yaml b/tests/v0.3/core/path-fields.yaml index bcc7711..35ad080 100644 --- a/tests/v0.3/core/path-fields.yaml +++ b/tests/v0.3/core/path-fields.yaml @@ -865,6 +865,54 @@ groups: code: path_value_missing field: title + - name: "a create that normalizes a stored value reports path_value_normalized" + id: path_fields.create_normalized_warning + since: 0.3.0-rc.6 + covers: [core_write.path_value_normalized] + operation: create + input: + type: task + frontmatter: { title: "Plan: Q3/Q4", status: open } + expect: + valid: true + path: "tasks/Plan Q3Q4.md" + effective_frontmatter: { title: "Plan Q3Q4" } + diagnostics: + - code: path_value_normalized + severity: warning + field: title + details: { written: "Plan: Q3/Q4", stored: "Plan Q3Q4" } + + - name: "an update whose value normalizes to the current one warns and does not move" + id: path_fields.update_normalized_warning + since: 0.3.0-rc.6 + covers: [core_write.path_value_normalized] + operation: update + input: + path: "tasks/Call Bob.md" + patch: { title: "Call Bob?" } + expect: + valid: true + path: "tasks/Call Bob.md" + diagnostics: + - code: path_value_normalized + severity: warning + field: title + details: { written: "Call Bob?", stored: "Call Bob" } + + - name: "a value that normalization leaves unchanged reports nothing" + id: path_fields.unchanged_value_no_warning + since: 0.3.0-rc.6 + covers: [core_write.path_value_normalized] + operation: create + input: + type: task + frontmatter: { title: "Email Carol", status: open } + expect: + valid: true + path: "tasks/Email Carol.md" + diagnostics: [] + - name: "id and title in the filename" setup: config: | diff --git a/tests/v0.3/lifecycle/lifecycle.yaml b/tests/v0.3/lifecycle/lifecycle.yaml index 6fa93de..a875deb 100644 --- a/tests/v0.3/lifecycle/lifecycle.yaml +++ b/tests/v0.3/lifecycle/lifecycle.yaml @@ -268,3 +268,74 @@ groups: valid: true frontmatter: stale: true + + - name: "slugify" + setup: + config: | + spec_version: "0.3.0" + types: + page.md: | + --- + kind: mdbase.type + name: page + version: 1 + match: + path_glob: "pages/**/*.md" + schema: + dialect: json-schema-2020-12 + value: + type: object + properties: + title: { type: string } + slug: { type: string } + lifecycle: + on_create: + set: + slug: { slugify: title } + --- + files: {} + tests: + - name: "accents are removed by canonical decomposition and runs become one dash" + id: lifecycle.slugify_accents + since: 0.3.0-rc.6 + covers: [lifecycle.slugify] + operation: create + input: + type: page + path: "pages/a.md" + frontmatter: + title: " Café — Crème Brûlée!! " + expect: + valid: true + frontmatter_contains: + slug: cafe-creme-brulee + + - name: "characters that do not decompose to ASCII are separators, not transliterated" + id: lifecycle.slugify_no_transliteration + since: 0.3.0-rc.6 + covers: [lifecycle.slugify] + operation: create + input: + type: page + path: "pages/b.md" + frontmatter: + title: "Straße Ærø 2" + expect: + valid: true + frontmatter_contains: + slug: stra-e-r-2 + + - name: "a value without ASCII letters or digits gives the empty string" + id: lifecycle.slugify_empty + since: 0.3.0-rc.6 + covers: [lifecycle.slugify] + operation: create + input: + type: page + path: "pages/c.md" + frontmatter: + title: "日本語" + expect: + valid: true + frontmatter_contains: + slug: "" diff --git a/tests/v0.3/manifest.yaml b/tests/v0.3/manifest.yaml index 715fa96..28f68d0 100644 --- a/tests/v0.3/manifest.yaml +++ b/tests/v0.3/manifest.yaml @@ -34,6 +34,7 @@ claim_profiles: - invalid_records_reported # since 0.3.0-rc.5 - path_equivalence # since 0.3.0-rc.5 - regex_profile # since 0.3.0-rc.5 + - frontmatter_delimiters # since 0.3.0-rc.6 - id: collection_semantics status: coverage_complete requires: [core_read] @@ -94,6 +95,8 @@ claim_profiles: - named_projections - grouping_and_summaries - query_evaluation_diagnostics + - value_order # since 0.3.0-rc.6 + - path_tiebreak # since 0.3.0-rc.6 - id: links status: draft requires: [collection_semantics, cel] @@ -105,6 +108,9 @@ claim_profiles: - id_field_no_default # since 0.3.0-rc.5 - ambiguous_id_link # since 0.3.0-rc.5 - filename_tiebreakers # since 0.3.0-rc.5 + - path_key_matching # since 0.3.0-rc.6 + - markdown_destinations # since 0.3.0-rc.6 + - body_code_and_tags # since 0.3.0-rc.6 - id: core_write status: draft requires: [collection_semantics] @@ -120,6 +126,8 @@ claim_profiles: - rename_references # since 0.3.0-rc.5 - body_edits # since 0.3.0-rc.5 - path_stored_writes # since 0.3.0-rc.6 + - path_value_normalized # since 0.3.0-rc.6 + - create_membership_path # since 0.3.0-rc.6 - id: merge # since 0.3.0-rc.5 status: coverage_complete requires: [core_write] @@ -155,6 +163,7 @@ claim_profiles: requires: [core_write, cel] requirements: - ordered_actions + - slugify # since 0.3.0-rc.6 - id: event_action_interop/0.1 status: draft requires: [] @@ -254,6 +263,7 @@ fixture_sets: - cel/cel-profile.yaml - cel/match-determinism.yaml - cel/regex-profile.yaml + - cel/query-order.yaml - id: views description: Ordinary view records resolve named queries, invocation context, and advisory presentation. coverage_targets: [cel_query] From 968dec74c5750be8bc52c6e126b1dbcb93ac04c9 Mon Sep 17 00:00:00 2001 From: callumalpass Date: Thu, 12 Nov 2026 07:01:04 +1100 Subject: [PATCH 2/7] Move type packs into the type-packs/0.1 companion profile The pack manifest, lock file, assessment, and apply sections move verbatim from Chapter 05A. The profile ID is now type_packs/0.1, in the manifest, claim schema, checker, and fixtures. --- 00-overview.md | 4 +- 02-collection-layout.md | 3 +- 05-type-files.md | 3 +- 05a-data-contracts.md | 269 +-------------- 14-conformance.md | 48 +-- release/v0.3.0.md | 6 +- runtime/0.2.md | 8 +- schemas/v0.3/conformance-claim.schema.json | 4 +- scripts/check_v03_tests.py | 2 +- site/build.mjs | 2 + .../valid-all-profile-dependencies.yml | 2 +- tests/v0.3/manifest.yaml | 4 +- tests/v0.3/type-packs/type-packs.yaml | 52 +-- type-packs/0.1.md | 312 ++++++++++++++++++ 14 files changed, 390 insertions(+), 329 deletions(-) create mode 100644 type-packs/0.1.md diff --git a/00-overview.md b/00-overview.md index 2963fd3..9e6c88d 100644 --- a/00-overview.md +++ b/00-overview.md @@ -313,7 +313,7 @@ keep those claims precise and independently testable. | [03-records-and-frontmatter.md](./03-records-and-frontmatter.md) | Markdown parsing and YAML value semantics | | [04-configuration.md](./04-configuration.md) | The `mdbase.yaml` configuration file | | [05-type-files.md](./05-type-files.md) | JSON Schema type wrappers and metadata | -| [05a-data-contracts.md](./05a-data-contracts.md) | versioned data contracts, type implementations, and transactional type packs | +| [05a-data-contracts.md](./05a-data-contracts.md) | versioned data contracts and type implementations | | [06-json-schema-profile.md](./06-json-schema-profile.md) | Supported JSON Schema vocabulary and reference rules | | [07-collection-semantics.md](./07-collection-semantics.md) | Matching, defaults, uniqueness, links, and paths | | [08-links.md](./08-links.md) | Link syntax, resolution, traversal, and backlinks | @@ -330,6 +330,7 @@ chapters: | Document | Description | | --- | --- | +| [type-packs/0.1.md](./type-packs/0.1.md) | Type packs profile 0.1: pack manifests, the lock file, assessment, and transactional apply | | [interop/0.1.md](./interop/0.1.md) | Event and action interoperability profile 0.1 | | [runtime/0.2.md](./runtime/0.2.md) | Durable runtime companion profile 0.2: standard pack, authorization, admission, execution, and recovery | | [adapters/obsidian-bases.md](./adapters/obsidian-bases.md) | Obsidian Bases saved-view adapter | @@ -353,6 +354,7 @@ Companion profiles, artifacts, and schemas carry their own versions: | Versioned thing | Declared by | Current | Format | | --- | --- | --- | --- | | collection specification | `spec_version` in `mdbase.yaml` | `0.3.0` | semantic version | +| type packs profile | profile ID `type_packs/0.1` | `0.1` | major.minor | | event/action interoperability profile | conformance claim `interop_profile_version` | `0.1` | major.minor | | durable runtime profile | conformance claim `runtime_profile_version` | `0.2` | major.minor | | interoperability testbed protocol | testbed evidence | `0.1` | major.minor | diff --git a/02-collection-layout.md b/02-collection-layout.md index 4e37ba0..f2cabc6 100644 --- a/02-collection-layout.md +++ b/02-collection-layout.md @@ -45,7 +45,8 @@ record scan unless explicitly excluded. The following paths are reserved by default: - `mdbase.yaml` -- `mdbase.lock.yaml`, when managed type packs are installed +- `mdbase.lock.yaml`, when managed + [type packs](./type-packs/0.1.md) are installed - the configured types folder, default `_types/` - the configured contracts folder, default `_contracts/` - `.mdbase/` for derived implementation state diff --git a/05-type-files.md b/05-type-files.md index 93b95fb..3cd9c61 100644 --- a/05-type-files.md +++ b/05-type-files.md @@ -196,7 +196,8 @@ implementing type matches `type: view` and maps every contract field to the same-named frontmatter field. A collection that stores portable view records SHOULD install that contract and -a type implementing it, normally through the `mdbase.view` type pack. A +a type implementing it, normally through the `mdbase.view` +[type pack](./type-packs/0.1.md). A collection MAY instead implement the contract with its own type name, match rule, and field mapping. diff --git a/05a-data-contracts.md b/05a-data-contracts.md index a5325c9..6853aa1 100644 --- a/05a-data-contracts.md +++ b/05a-data-contracts.md @@ -27,8 +27,10 @@ The complete collection contract model has three intentionally small parts: interface using JSON Schema 2020-12. 2. For a `record` contract, a type's `implements` entry maps that interface to the type and supplies contract-specific binding data. -3. An optional managed `mdbase.type-pack` groups contracts, types, and their - referenced schemas for transactional installation and evolution. +3. An optional `mdbase.type-pack` groups contracts, types, and their + referenced schemas for transactional installation and evolution. It is + defined by the separately versioned + [type packs profile](./type-packs/0.1.md). Event sources and action providers make runtime declarations because they are executable, instance-specific implementations rather than record types. @@ -159,7 +161,8 @@ Different content for the same ID and version is Resolution is collection-local and offline. Core implementations MUST NOT fetch a missing contract from the network. Applications and installers carry the -contract files they require, usually in a type pack. +contract files they require, usually in a +[type pack](./type-packs/0.1.md). ## Type Implementations @@ -341,263 +344,6 @@ Core collection APIs remain authorization-neutral. This distinction is normative for gateways and application protocols that use data contracts as an authorization boundary. -## Type Packs - -A type pack is a directory or archive with an `mdbase-pack.yaml` manifest that -validates against `schemas/v0.3/type-pack.schema.json`. - -```yaml -kind: mdbase.type-pack -id: example.tasks -version: 1.0.0 -name: Example task types -resources: - - kind: contract - mode: managed - source: contracts/example.task.md - target: _contracts/example.task.md - digest: sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef - - kind: type - mode: managed - source: types/task.md - target: _types/task.md - digest: sha256:abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789 -``` - -`mode` is required. A `managed` resource remains owned by the pack and may be -updated or retired by a later pack version only while its live bytes still -match the installed digest. A `seed` resource is created only when its target -is absent and becomes user-owned immediately; later pack versions neither -replace nor delete it unless an explicit seed-type upgrade is declared below. - -A seed **type** resource MAY declare `upgrade_from`: one baseline, or a -non-empty list of baselines. A baseline is `{ digest, document }` with an -optional integer `version`. `document` is the exact bytes of a starter the -publisher previously shipped for this resource, pinned by SHA-256 in `digest`; -it is not an assertion that the user's current document is unchanged. `version`, -when present, is the `version` that document declares and is only for -presentation. A single baseline is equivalent to a list containing it. Listing -every previously shipped starter that remains supported lets a collection -upgrade from any of them, not only from the most recent. This declaration is -part of the reviewed manifest and its digest. Ordinary seeds are unaffected. -Engines that do not support this member MUST reject the manifest. - -The manifest is invalid (`invalid_type_pack`) when `upgrade_from` appears on a -resource that is not a seed type, when it is an empty list, when a baseline lacks -`digest` or `document`, when a baseline's digest is not the SHA-256 of -its document, when two baselines share a digest, when a baseline's digest equals -the resource's own digest, when a baseline document's frontmatter `kind` or -`name` differs from the desired document's, or when a baseline's `version` -differs from the `version` its document's frontmatter declares. - -A seed's **origin** is the publisher document its live target descends from. -The lock records it (see Pack Identity And Portable Provenance). When the target -exists, is not an intentionally preserved seed target, and the resource declares -`upgrade_from`, the engine plans exactly one of these, in order: - -Two type documents are **equal** when their frontmatter values are equal and -their Markdown bodies are identical; YAML formatting and key order do not -matter, because other writers may re-serialize a type file. - -1. The live type equals the desired document: `preserve`. -2. The live type equals a baseline's document: `update`, writing the desired - document byte-for-byte. Equality proves the origin, whatever the lock - records. -3. The origin is the desired document: `preserve`. The seed was already upgraded - and has been edited since. -4. The origin is a baseline: `update` by a three-way merge of that baseline, the - live type, and the desired type. -5. Otherwise the origin is unknown (a lock written before origins were - recorded, or none) or is not a listed baseline, and the engine **infers** a - merge base. It orders the listed baselines by how many frontmatter leaf - values differ from the live type's (a key present on one side only counts - once; equal-length arrays compare item by item; any other differing value - counts once), fewest first and in listed order on a tie. It takes the first - baseline whose three-way merge succeeds and whose merged type validates and - implements every exact contract version the desired type does, judged over - the pack's own contract resources: `update` with that merge, or `preserve` - when the merge changes nothing. -6. Otherwise, when the desired type declares `implements` and the live type - already validates and implements every exact contract version the desired - type does: `preserve`. The seed already fits the pack. -7. Otherwise: `preserve`, with a `reason` stating that no upgrade baseline - applies. The type is left as it is; it does not make the pack conflict. - -In cases 1, 2, 3, and 5 the lock's origin becomes the desired document; in the -others it is carried forward. An engine MUST NOT choose a merge baseline any -other way. An inferred base (case 5) may not be the seed's true origin, so the -differences between them could be applied as if they were the user's edits; the -conservative merge bounds this, since a seed whose `version` or other setting -differs from both sides is a competing change and fails. The assessment -reports, for every seed `update`, the baseline used as -`upgrade_baseline: { digest, version? }`, so the change is reviewed like any -other. - -A three-way merge is conservative. The type kind and name MUST match. Unchanged publisher -settings retain live customizations; unchanged live settings accept publisher -changes. Object settings merge recursively. Contract implementations merge by -contract ID only when unambiguous, retaining customized field mappings and -bindings while updating exact version references. Competing changes (including -ambiguous implementation lists) MUST fail closed. Other arrays merge item by -item when the baseline, live, and desired arrays have the same length, and are -otherwise indivisible. -Missing values and explicit null are distinct. Top-level setting removal -requires separate manual review. The live Markdown body and unrelated YAML -nodes MUST be preserved. Reformatting a changed YAML node is permitted. - -The merged resource digest MUST appear in the assessment. Normal assessment -and collection revision guards still apply. The upgrade, contract resources, -and provenance MUST publish atomically only after validating the staged -collection. A remaining reference to a removed exact contract MUST block the -whole upgrade; implementations MUST NOT rewrite additional types implicitly. -Record documents MUST NOT be migrated by this mechanism. An intentionally -preserved seed target MUST NOT be upgraded. - -This is not version negotiation or grant migration. Callers coordinating -applications must retain old contracts and implementations when supported -applications still require them, or arrange a separately reviewed coordinated -upgrade. Existing grants never acquire the new contract version implicitly. - -Resource digests are SHA-256 over the exact resource bytes. Source and target -paths are relative, forward-slash paths without traversal. -A directory, archive, repository, or package that distributes a pack MUST -preserve those bytes exactly, including line endings. Rewriting a text resource -from LF to CRLF therefore creates a different resource and MUST fail digest -verification. Distributors that store packs in version control SHOULD disable -line-ending conversion for pack resources. - -Type packs are installation units, not record types and not permission grants. -A pack may include several contracts, several implementing or auxiliary types, -and local JSON Schemas referenced by those artifacts. - -## Pack Identity And Portable Provenance - -The pack digest is SHA-256 over RFC 8785 JSON Canonicalization Scheme bytes for -the complete validated manifest, prefixed with `sha256:`. Because every -resource digest is part of the manifest, the pack digest identifies the exact -manifest and exact resource bytes without incorporating their distribution -location. - -Managed pack state is stored in `mdbase.lock.yaml` at the collection root and -validates against `schemas/v0.3/type-pack-lock.schema.json`. The lock is -portable collection source, not derived cache state. It records each installed -pack's exact ID, version, pack digest, stable installer identity, and the kind, -mode, canonical source, resolved target, and installed digest of every resource. Tools write it -deterministically and users MAY inspect or version it. Tools MUST NOT infer -ownership from filenames, application names, or `x-*` metadata. - -A seed resource's entry also records `origin_digest`, the digest of the -publisher document its target descends from, when that is known. Because a seed -becomes user-owned, the installed digest of a seed describes the pack, not the -target, and MUST NOT be used as its origin. An apply sets `origin_digest` to the -desired resource digest when it creates the seed, when it upgrades it (by exact -replacement or by merge), and whenever the target's live bytes equal the desired -document, whatever else applies. Otherwise it carries the previous entry's -`origin_digest` for that target forward unchanged, or omits it when there is -none. A seed target that existed before the pack was installed, and an -intentionally preserved seed target, therefore have no origin unless their bytes -equal the desired document. A lock entry without `origin_digest` means the -origin is unknown. Locks written before `origin_digest` existed carry none, so -an edited seed under such a lock is preserved with a reason rather than merged -until it is upgraded or recreated; an unedited seed still upgrades, because its -bytes prove its origin. `origin_digest` is defined only for seed resources and -MUST NOT appear on a managed resource. An apply that changes only seed origins leaves the pack's -status `current`; it is not a reconfiguration. - -Full collection snapshots, authority transfers, and unscoped synchronization -MUST carry `mdbase.lock.yaml` when it exists. A scoped application projection -MAY omit it to avoid disclosing unrelated pack metadata. A storage provider -MUST NOT reject the lock under a per-document limit that the collection owner -cannot remediate, because tools generate it. - -An absent lock means no resource is pack-managed, including resources created -by older one-shot installers. Adopting existing files into managed ownership is -a separate explicit operation and MUST verify their exact digests. - -## Assessment And Transactional Apply - -Pack evolution has two public phases: `assess_type_pack` is read-only and -`apply_type_pack` consumes a reviewed assessment. A caller supplies the desired -manifest, exact resources, and a stable reverse-domain `installed_by` -application or tool identity. Optional `target_overrides` map canonical manifest -targets to collection-specific paths without changing the publisher-owned pack -identity. Overrides are validated as safe collection paths, are included in the -assessment digest, and the resolved targets are written to the receipt. - -Assessment reports one of `current`, `install`, `upgrade`, `downgrade`, -`reconfigure`, or `conflict`, the current and desired pack identities, and every resource in a -stable exact diff. Resource actions are `create`, `update`, `delete`, -`adopt`, `unchanged`, `preserve`, or `conflict`. `adopt` records an existing -byte-identical managed resource without rewriting it. `preserve` is used for seed resources -and for seed resources retired from a newer pack. A conflicting assessment is -not applicable. The assessment also reports the planned `mdbase.lock.yaml` -action (`create`, `update`, or `unchanged`) and its resulting digest, so storage -adapters can persist the complete authority transaction without inferring -hidden engine writes. - -`reconfigure` means the exact same immutable pack is being resolved to different -collection targets. Managed resources are relocated only when their installed -bytes are unchanged; modified resources conflict. Seed resources are never -moved or deleted automatically. A caller may name seed targets in -`preserve_seed_targets` to record an intentional omission, such as when a user -maps the contract to an existing type instead of accepting the starter. Unknown -or non-seed preservation targets MUST be rejected. - -When guided setup maps a provided contract to an existing user-owned type, the -reviewed `contract_setups` choices are part of the same assessment and apply. -The assessment reports their exact type-resource diff and binds the selected -type revisions, field mappings, and bindings into `assessment_digest`. Apply -MUST stage those edits together with the managed resources and lock, validate -the combined collection, and commit them in the same transaction. A stale type -revision therefore leaves both the user-owned type and every pack resource -unchanged. A connector MUST NOT install the pack first and apply the reviewed -mapping as a later independent mutation. - -The assessment includes a deterministic `assessment_digest` over the desired -pack identity, current lock entry, and every relevant live target digest. Apply -MUST recompute it inside the collection mutation boundary and fail with -`concurrent_modification` before writing when it differs. Downgrades require an -explicit `allow_downgrade` decision. - -An unmanaged target with different bytes is a conflict unless the assessment -request contains an explicit adoption decision naming that target and its exact -current digest. An accepted adoption may plan `update`, records the adopted -digest in the diff, and is covered by `assessment_digest`. This is the safe -upgrade bridge for collections created before pack receipts existed: tools can -show the files that will become package-managed, require informed approval, and -remain race-safe. Unknown targets, seed targets, and stale adoption digests MUST -be rejected. Adoption is never inferred from application access or install -intent. - -A pack-aware apply implementation MUST: - -1. validate the manifest, safe paths, source bytes, and resource digests -2. stage every resource and reviewed existing-type setup without changing the live collection -3. resolve the complete staged contract and type registries -4. validate every contract, implementation, type, reference, and affected - existing record -5. compute and present the exact resource diff and assessment digest -6. acquire its collection mutation boundary and recheck the assessment -7. commit all resource changes and the lock as one recoverable transaction -8. reopen the collection and verify the committed registry and lock - -An invalid resource aborts before any live write. A conflict or concurrent -change aborts with the live collection unchanged. Implementations may use an -atomic directory exchange or a durable backup journal with rollback. After a -crash, recovery MUST restore either the complete pre-install state or the -complete committed state before normal collection operations resume. - -Revoking an application's access does not uninstall its type pack. Uninstall is -a separate, explicitly requested operation because records may still depend on -the installed types. - -Applying a current pack is valid and reports every managed resource as -`unchanged`; it MUST NOT create a new logical collection revision. Applying an -upgrade deletes a retired managed target only when its live digest still -matches the lock. User-modified managed resources produce `conflict` and no -live write. Successful apply reports the committed receipt, exact resource -diff, and `cleanup_deferred` when transaction-journal cleanup must be retried. - ## Diagnostics Data-contract-aware tools use these codes: @@ -611,9 +357,6 @@ Data-contract-aware tools use these codes: | `data_contract_binding_invalid` | implementation binding fails its binding schema | | `data_contract_field_invalid` | a mapped contract or record field is missing or incompatible | | `data_contract_record_invalid` | a projected contract view fails the contract schema | -| `invalid_type_pack` | a pack manifest, lock, resource, path, or digest is invalid | -| `type_pack_conflict` | ownership, live bytes, or an untracked target prevents a safe apply | -| `type_pack_apply_failed` | transactional commit or recovery did not complete normally | Diagnostics use the canonical shape from Chapter 14 and identify the contract ID, exact version, type name, and relevant field mapping in `details`. diff --git a/14-conformance.md b/14-conformance.md index 483824c..741a9cb 100644 --- a/14-conformance.md +++ b/14-conformance.md @@ -17,13 +17,18 @@ queries, writes, runtime preflight, workflow execution, and watching. | Links | parse, resolve, validate, and traverse links | | Core Write | create, update, delete, rename, and batch records | | Merge | merge concurrent versions of a record with declared field strategies | -| Type Packs | assess and transactionally apply managed type packs | | Lifecycle | apply standard managed-field policy during writes | -| Event/Action Interoperability | exchange CloudEvents and admitted action invocations through independently claimable roles | -| Durable Runtime 0.2 | validate standard runtime records, admit exact plans through interoperability declarations, and execute/recover them durably | | Watch | report ordered collection changes after consistent state | -Normative profile IDs and dependencies are: +Companion profiles add optional behavior on top of these: + +| Companion profile | Purpose | +| --- | --- | +| Type Packs 0.1 | assess and transactionally apply managed type packs | +| Event/Action Interoperability 0.1 | exchange CloudEvents and admitted action invocations through independently claimable roles | +| Durable Runtime 0.2 | validate standard runtime records, admit exact plans through interoperability declarations, and execute/recover them durably | + +Normative profile IDs and dependencies of this specification are: | Profile ID | Requires | | --- | --- | @@ -36,12 +41,18 @@ Normative profile IDs and dependencies are: | `links` | `collection_semantics`, `cel` | | `core_write` | `collection_semantics` | | `merge` | `core_write` | -| `type_packs` | `data_contracts`, `core_write` | | `lifecycle` | `core_write`, `cel` | -| `event_action_interop/0.1` | none | -| `runtime/0.2` | `data_contracts`, `event_action_interop/0.1`, `cel` | | `watch` | `collection_semantics` | +Companion profiles are versioned independently of this specification, and +their profile IDs carry their version: + +| Profile ID | Requires | Defined by | +| --- | --- | --- | +| `type_packs/0.1` | `data_contracts`, `core_write` | [type packs profile](./type-packs/0.1.md) | +| `event_action_interop/0.1` | none | [event/action interoperability profile](./interop/0.1.md) | +| `runtime/0.2` | `data_contracts`, `event_action_interop/0.1`, `cel` | [durable runtime profile](./runtime/0.2.md) | + An implementation claims a profile after passing every required behavior and test for that profile. `optional_features` records additional work outside the verified profile list. @@ -163,7 +174,6 @@ The v0.3 core codes include `invalid_request`, `duplicate_batch_path`, `invalid_data_contract`, `data_contract_not_found`, `data_contract_conflict`, `data_contract_version_mismatch`, `data_contract_binding_invalid`, `data_contract_field_invalid`, `data_contract_record_invalid`, -`invalid_type_pack`, `type_pack_conflict`, `type_pack_apply_failed`, `invalid_timezone`, and the JSON Schema `schema_` codes from Chapter 06. Runtime profile 0.2 reuses interoperability codes such as `unknown_contract`, `contract_digest_conflict`, `no_provider`, @@ -207,22 +217,6 @@ Data Contracts implementations MUST: - return every conforming implementation of a contract rather than selecting one -## Type Packs Requirements - -Type Packs implementations MUST: - -- validate pack manifests, safe resource paths, and resource digests -- read and write `mdbase.lock.yaml` deterministically -- produce a read-only assessment with an exact resource diff and - `assessment_digest` -- honor managed and seed resource ownership, explicit adoption, and target - overrides -- recheck the assessment inside the mutation boundary and fail with - `concurrent_modification` when it changed -- commit resources, reviewed type setups, and the lock as one recoverable - transaction -- treat reapplying a current pack as a no-op - ## Collection Semantics Requirements Collection Semantics implementations MUST: @@ -439,6 +433,12 @@ Lifecycle implementations MUST: `type_membership_changed` - report conflicts between matched lifecycle policies +## Type Packs Requirements + +The `type_packs/0.1` profile's requirements, and the diagnostic codes +`invalid_type_pack`, `type_pack_conflict`, and `type_pack_apply_failed`, are +defined by the [type packs profile](./type-packs/0.1.md). + ## Event/Action Interoperability Requirements The `event_action_interop/0.1` profile, its independently claimable roles, and diff --git a/release/v0.3.0.md b/release/v0.3.0.md index 969dfca..b6cf9a5 100644 --- a/release/v0.3.0.md +++ b/release/v0.3.0.md @@ -16,7 +16,7 @@ promoted to the repository root. The fourth release candidate applies the specification review: standard CEL semantics with string dates, `patch` and `unset` updates, atomic batches, validation levels, portable globs and built-in exclusions, filename-first link resolution, version requirements, the -`data_contracts` and `type_packs` profiles, and configuration migration rules. +`data_contracts` profile, the `type_packs/0.1` companion profile, and configuration migration rules. Stable remains blocked on registry install verification and completion of downstream multi-platform qualification. @@ -105,9 +105,9 @@ passes preflight. | `mdbase-spec` | normative prose, schemas, fixtures, migration rules | all artifact and site gates | source of truth | | `@callumalpass/mdbase-runtime` | durable admission and runtime record validation built on interoperability | `runtime/0.2`; browser package smoke | no TaskNotes/Obsidian/Node dependency in main export | | `mdbase-runtime-contracts-rs` | independent runtime contract validation harness | canonical schema, example collection, and embedded-schema tests | consume the same versioned artifacts as the TypeScript runtime package | -| `@callumalpass/mdbase` | TypeScript collection and data-contract implementation | `core_read`, `collection_semantics`, `data_contracts`, `cel`, `cel_query`, `links`, `core_write`, `type_packs`, `lifecycle` | exact compatible spec artifacts | +| `@callumalpass/mdbase` | TypeScript collection and data-contract implementation | `core_read`, `collection_semantics`, `data_contracts`, `cel`, `cel_query`, `links`, `core_write`, `type_packs/0.1`, `lifecycle` | exact compatible spec artifacts | | native `mdbase` CLI | safe user-facing operations, migration, direct filesystem access, Connect-managed collection access, service management, and profiling | packaged native binaries plus v0.2 migration rehearsals | assembled by `mdbase-connect` from `mdbase-rs`; the former TypeScript `mdbase-cli` is retired | -| `mdbase-rs` | independent Rust collection implementation and separately versioned workflow engine | `core_read`, `collection_semantics`, `data_contracts`, `cel`, `cel_match`, `cel_query`, `links`, `core_write`, `type_packs`, `lifecycle`, `runtime/0.2`, `watch`; workflow execution remains separately evidenced | claim only verified profiles per artifact | +| `mdbase-rs` | independent Rust collection implementation and separately versioned workflow engine | `core_read`, `collection_semantics`, `data_contracts`, `cel`, `cel_match`, `cel_query`, `links`, `core_write`, `type_packs/0.1`, `lifecycle`, `runtime/0.2`, `watch`; workflow execution remains separately evidenced | claim only verified profiles per artifact | | `mdbase-lsp` | editor diagnostics from Rust semantics | Rust claim plus LSP diagnostic fixtures | no independent schema dialect | | `mdbase-obsidian` | selected Obsidian host adapter and collection UI | shared-host, unload, and provider-discovery smoke tests | owns one generic host per loaded vault | | `@tasknotes/model` | TaskNotes data contract, type pack, domain types, and provider metadata | contract conformance plus ESM/CJS package smoke | generic runtime types come from mdbase runtime package | diff --git a/runtime/0.2.md b/runtime/0.2.md index 7a23368..3834476 100644 --- a/runtime/0.2.md +++ b/runtime/0.2.md @@ -7,8 +7,8 @@ recoverable automation. It is deliberately a **companion** to two simpler specifications: 1. the core mdbase contract system defines contract identity, semantic - versions, JSON Schemas, digests, record implementations, resolution, and - packs; + versions, JSON Schemas, digests, record implementations, and resolution, + and the [type packs profile](../type-packs/0.1.md) defines packs; 2. the [event/action interoperability profile](../interop/0.1.md) defines CloudEvents, action requests and outcomes, event-source and action-provider declarations, provider selection, and baseline exchange behavior; @@ -269,8 +269,8 @@ collection, but loading the collection remains safe and does not activate them. ## Conformance Boundary A conformant core engine supports contract artifacts, type implementations, -record projection, conflict detection, and packs. It need not execute -workflows. +record projection, and conflict detection. It need not execute workflows or +install packs. A conformant interoperability implementation supports its claimed event/action roles. It need not persist runs. diff --git a/schemas/v0.3/conformance-claim.schema.json b/schemas/v0.3/conformance-claim.schema.json index f66cf4e..69cef6a 100644 --- a/schemas/v0.3/conformance-claim.schema.json +++ b/schemas/v0.3/conformance-claim.schema.json @@ -187,7 +187,7 @@ "then": { "properties": { "profiles": { "contains": { "const": "core_read" } } } } }, { - "if": { "properties": { "profiles": { "contains": { "const": "type_packs" } } } }, + "if": { "properties": { "profiles": { "contains": { "const": "type_packs/0.1" } } } }, "then": { "properties": { "profiles": { @@ -262,7 +262,7 @@ "links", "core_write", "merge", - "type_packs", + "type_packs/0.1", "lifecycle", "event_action_interop/0.1", "runtime/0.2", diff --git a/scripts/check_v03_tests.py b/scripts/check_v03_tests.py index c4334c0..28cb603 100755 --- a/scripts/check_v03_tests.py +++ b/scripts/check_v03_tests.py @@ -79,7 +79,7 @@ "links", "core_write", "merge", - "type_packs", + "type_packs/0.1", "lifecycle", "event_action_interop/0.1", "runtime/0.2", diff --git a/site/build.mjs b/site/build.mjs index cd6d3ba..47df560 100644 --- a/site/build.mjs +++ b/site/build.mjs @@ -89,6 +89,7 @@ const SPEC_FILES = [ { file: '12a-concurrent-edits.md', num: '12A', title: 'Concurrent Edits', id: 'section-12a' }, { file: '13-migrations-and-compatibility.md', num: '13', title: 'Migrations & Compatibility', id: 'section-13' }, { file: '14-conformance.md', num: '14', title: 'Conformance', id: 'section-14' }, + { file: 'type-packs/0.1.md', num: 'P', title: 'Type Packs', id: 'companion-type-packs', group: 'Companion Profiles' }, { file: 'interop/0.1.md', num: 'I', title: 'Event & Action Interoperability', id: 'companion-interop', group: 'Companion Profiles' }, { file: 'runtime/0.2.md', num: 'R', title: 'Durable Runtime', id: 'companion-runtime', group: 'Companion Profiles' }, { file: 'adapters/obsidian-bases.md', num: 'O', title: 'Obsidian Bases Adapter', id: 'companion-obsidian-bases', group: 'Companion Profiles' }, @@ -188,6 +189,7 @@ renderer.link = function ({ href, title, text }) { } const companions = { + 'type-packs/0.1.md': '#companion-type-packs', 'interop/0.1.md': '#companion-interop', 'runtime/0.2.md': '#companion-runtime', 'adapters/obsidian-bases.md': '#companion-obsidian-bases', diff --git a/tests/v0.3/fixtures/conformance/valid-all-profile-dependencies.yml b/tests/v0.3/fixtures/conformance/valid-all-profile-dependencies.yml index 2f7086f..eee193e 100644 --- a/tests/v0.3/fixtures/conformance/valid-all-profile-dependencies.yml +++ b/tests/v0.3/fixtures/conformance/valid-all-profile-dependencies.yml @@ -16,7 +16,7 @@ profiles: - cel_query - links - core_write - - type_packs + - type_packs/0.1 - lifecycle - event_action_interop/0.1 - runtime/0.2 diff --git a/tests/v0.3/manifest.yaml b/tests/v0.3/manifest.yaml index 28f68d0..8a6c0b0 100644 --- a/tests/v0.3/manifest.yaml +++ b/tests/v0.3/manifest.yaml @@ -144,7 +144,7 @@ claim_profiles: - merge_format_fidelity - path_merge - body_edit_rebase # since 0.3.0-rc.5 - - id: type_packs + - id: type_packs/0.1 status: draft requires: [data_contracts, core_write] requirements: @@ -248,7 +248,7 @@ fixture_sets: - data-contracts/data-contracts.yaml - id: type_packs description: Complete type packs validate, report exact diffs, install atomically, reinstall idempotently, and upgrade seed types from their recorded origin. - coverage_targets: [type_packs] + coverage_targets: [type_packs/0.1] files: - type-packs/type-packs.yaml - id: lifecycle diff --git a/tests/v0.3/type-packs/type-packs.yaml b/tests/v0.3/type-packs/type-packs.yaml index 1ee73b1..0f44532 100644 --- a/tests/v0.3/type-packs/type-packs.yaml +++ b/tests/v0.3/type-packs/type-packs.yaml @@ -30,11 +30,11 @@ groups: lock_exists: true implementations: 1 covers: - - type_packs.transactional_type_pack_preflight - - type_packs.exact_type_pack_diff - - type_packs.idempotent_type_pack_apply - - type_packs.managed_type_pack_provenance - - type_packs.managed_type_pack_assessment + - type_packs/0.1.transactional_type_pack_preflight + - type_packs/0.1.exact_type_pack_diff + - type_packs/0.1.idempotent_type_pack_apply + - type_packs/0.1.managed_type_pack_provenance + - type_packs/0.1.managed_type_pack_assessment - name: "preflight failure is atomic" setup: @@ -55,7 +55,7 @@ groups: code: invalid_type_pack targets_exist: [false, false] covers: - - type_packs.type_pack_atomicity + - type_packs/0.1.type_pack_atomicity - name: "target conflicts are atomic" setup: @@ -78,7 +78,7 @@ groups: code: type_pack_conflict targets_exist: [true, false] covers: - - type_packs.type_pack_atomicity + - type_packs/0.1.type_pack_atomicity - name: "managed upgrade conflict and concurrency" setup: @@ -99,7 +99,7 @@ groups: applicable: false actions: [conflict, unchanged] covers: - - type_packs.managed_type_pack_assessment + - type_packs/0.1.managed_type_pack_assessment - id: type-pack-stale-assessment name: "apply rejects a target changed after assessment without writing" operation: apply_type_pack @@ -111,7 +111,7 @@ groups: error: code: concurrent_modification covers: - - type_packs.managed_type_pack_concurrency + - type_packs/0.1.managed_type_pack_concurrency - name: "explicit adoption of legacy definitions" setup: @@ -138,7 +138,7 @@ groups: lock_exists: true implementations: 1 covers: - - type_packs.managed_type_pack_adoption + - type_packs/0.1.managed_type_pack_adoption - name: "seed type upgrades follow the seed's origin" setup: @@ -173,7 +173,7 @@ groups: lock_origin: _types/note.md: "examples/v0.3/seed-upgrades/v3/_types/note.md" covers: - - type_packs.seed_type_upgrade + - type_packs/0.1.seed_type_upgrade - id: seed-upgrade-single-baseline-object name: "a single baseline object is a list of one" @@ -195,7 +195,7 @@ groups: target_matches_source: _types/note.md: "examples/v0.3/seed-upgrades/v2/_types/note.md" covers: - - type_packs.seed_type_upgrade + - type_packs/0.1.seed_type_upgrade - id: seed-upgrade-edited-merges-against-origin name: "an edited seed merges against the baseline it was installed from, keeping the user's edits" @@ -234,7 +234,7 @@ groups: lock_origin: _types/note.md: "examples/v0.3/seed-upgrades/v3/_types/note.md" covers: - - type_packs.seed_type_upgrade + - type_packs/0.1.seed_type_upgrade - id: seed-upgrade-origin-survives-preserve name: "a pack version that preserves a seed keeps its origin, so a later upgrade merges against the true baseline" @@ -264,7 +264,7 @@ groups: /schema/value/properties/created: { type: string, format: date-time } /schema/value/properties/mood: { type: string } covers: - - type_packs.seed_type_upgrade + - type_packs/0.1.seed_type_upgrade - id: seed-upgrade-unlisted-origin-preserved name: "an edited seed whose origin is not a listed baseline is preserved with a reason, without a conflict" @@ -291,7 +291,7 @@ groups: lock_origin: _types/note.md: "examples/v0.3/seed-upgrades/v1/_types/note.md" covers: - - type_packs.seed_type_upgrade + - type_packs/0.1.seed_type_upgrade - id: seed-upgrade-edited-after-upgrade-preserved name: "a seed edited after its upgrade is preserved without a reason" @@ -315,7 +315,7 @@ groups: action: preserve reason: false covers: - - type_packs.seed_type_upgrade + - type_packs/0.1.seed_type_upgrade - name: "a seed target that existed before installation has no origin" setup: @@ -358,7 +358,7 @@ groups: lock_origin: _types/note.md: absent covers: - - type_packs.seed_type_upgrade + - type_packs/0.1.seed_type_upgrade - name: "an edited seed with no recorded origin merges against the nearest baseline" setup: @@ -417,7 +417,7 @@ groups: lock_origin: _types/note.md: "examples/v0.3/seed-upgrades/v3/_types/note.md" covers: - - type_packs.seed_type_upgrade + - type_packs/0.1.seed_type_upgrade - name: "upgrade baselines are validated" setup: @@ -436,7 +436,7 @@ groups: error: code: invalid_type_pack covers: - - type_packs.seed_type_upgrade_validation + - type_packs/0.1.seed_type_upgrade_validation - id: seed-upgrade-invalid-self-baseline name: "invalid upgrade_from (self-baseline) rejects the manifest" operation: assess_type_pack @@ -447,7 +447,7 @@ groups: error: code: invalid_type_pack covers: - - type_packs.seed_type_upgrade_validation + - type_packs/0.1.seed_type_upgrade_validation - id: seed-upgrade-invalid-version-mismatch name: "invalid upgrade_from (version-mismatch) rejects the manifest" operation: assess_type_pack @@ -458,7 +458,7 @@ groups: error: code: invalid_type_pack covers: - - type_packs.seed_type_upgrade_validation + - type_packs/0.1.seed_type_upgrade_validation - id: seed-upgrade-invalid-name-mismatch name: "invalid upgrade_from (name-mismatch) rejects the manifest" operation: assess_type_pack @@ -469,7 +469,7 @@ groups: error: code: invalid_type_pack covers: - - type_packs.seed_type_upgrade_validation + - type_packs/0.1.seed_type_upgrade_validation - id: seed-upgrade-invalid-managed-upgrade name: "invalid upgrade_from (managed-upgrade) rejects the manifest" operation: assess_type_pack @@ -480,7 +480,7 @@ groups: error: code: invalid_type_pack covers: - - type_packs.seed_type_upgrade_validation + - type_packs/0.1.seed_type_upgrade_validation - id: seed-upgrade-invalid-baseline-digest name: "invalid upgrade_from (baseline-digest) rejects the manifest" operation: assess_type_pack @@ -491,7 +491,7 @@ groups: error: code: invalid_type_pack covers: - - type_packs.seed_type_upgrade_validation + - type_packs/0.1.seed_type_upgrade_validation - id: seed-upgrade-invalid-empty-baselines name: "invalid upgrade_from (empty-baselines) rejects the manifest with invalid_type_pack" operation: assess_type_pack @@ -502,7 +502,7 @@ groups: error: code: invalid_type_pack covers: - - type_packs.seed_type_upgrade_validation + - type_packs/0.1.seed_type_upgrade_validation - id: seed-upgrade-invalid-incomplete-baseline name: "invalid upgrade_from (incomplete-baseline) rejects the manifest with invalid_type_pack" operation: assess_type_pack @@ -513,4 +513,4 @@ groups: error: code: invalid_type_pack covers: - - type_packs.seed_type_upgrade_validation + - type_packs/0.1.seed_type_upgrade_validation diff --git a/type-packs/0.1.md b/type-packs/0.1.md new file mode 100644 index 0000000..81b5b46 --- /dev/null +++ b/type-packs/0.1.md @@ -0,0 +1,312 @@ +# mdbase Type Packs Profile 0.1 + +## Purpose + +A type pack groups contracts, types, and the schemas they reference, so that +an application or installer can add them to a collection, and later upgrade +them, as one reviewed and recoverable change. + +This optional profile defines: + +- the `mdbase.type-pack` manifest and its resources; +- managed and seed resource ownership, and the seed-type upgrade; +- `mdbase.lock.yaml`, the portable record of what a pack installed; +- the read-only assessment and the transactional apply. + +The profile identifier is `type_packs/0.1`. It is versioned independently of +the mdbase collection specification `0.3.0` and requires the `data_contracts` +and `core_write` profiles of that specification. A tool that reads and writes +a collection, including the types and contracts that a pack installed, does +not need this profile: installed resources are ordinary type files, contract +files, and schemas. + +The normative schemas are `schemas/v0.3/type-pack.schema.json` and +`schemas/v0.3/type-pack-lock.schema.json`. + +## Pack Manifest + +A type pack is a directory or archive with an `mdbase-pack.yaml` manifest that +validates against `schemas/v0.3/type-pack.schema.json`. + +```yaml +kind: mdbase.type-pack +id: example.tasks +version: 1.0.0 +name: Example task types +resources: + - kind: contract + mode: managed + source: contracts/example.task.md + target: _contracts/example.task.md + digest: sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef + - kind: type + mode: managed + source: types/task.md + target: _types/task.md + digest: sha256:abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789 +``` + +`mode` is required. A `managed` resource remains owned by the pack and may be +updated or retired by a later pack version only while its live bytes still +match the installed digest. A `seed` resource is created only when its target +is absent and becomes user-owned immediately; later pack versions neither +replace nor delete it unless an explicit seed-type upgrade is declared below. + +A seed **type** resource MAY declare `upgrade_from`: one baseline, or a +non-empty list of baselines. A baseline is `{ digest, document }` with an +optional integer `version`. `document` is the exact bytes of a starter the +publisher previously shipped for this resource, pinned by SHA-256 in `digest`; +it is not an assertion that the user's current document is unchanged. `version`, +when present, is the `version` that document declares and is only for +presentation. A single baseline is equivalent to a list containing it. Listing +every previously shipped starter that remains supported lets a collection +upgrade from any of them, not only from the most recent. This declaration is +part of the reviewed manifest and its digest. Ordinary seeds are unaffected. +Engines that do not support this member MUST reject the manifest. + +The manifest is invalid (`invalid_type_pack`) when `upgrade_from` appears on a +resource that is not a seed type, when it is an empty list, when a baseline lacks +`digest` or `document`, when a baseline's digest is not the SHA-256 of +its document, when two baselines share a digest, when a baseline's digest equals +the resource's own digest, when a baseline document's frontmatter `kind` or +`name` differs from the desired document's, or when a baseline's `version` +differs from the `version` its document's frontmatter declares. + +A seed's **origin** is the publisher document its live target descends from. +The lock records it (see Pack Identity And Portable Provenance). When the target +exists, is not an intentionally preserved seed target, and the resource declares +`upgrade_from`, the engine plans exactly one of these, in order: + +Two type documents are **equal** when their frontmatter values are equal and +their Markdown bodies are identical; YAML formatting and key order do not +matter, because other writers may re-serialize a type file. + +1. The live type equals the desired document: `preserve`. +2. The live type equals a baseline's document: `update`, writing the desired + document byte-for-byte. Equality proves the origin, whatever the lock + records. +3. The origin is the desired document: `preserve`. The seed was already upgraded + and has been edited since. +4. The origin is a baseline: `update` by a three-way merge of that baseline, the + live type, and the desired type. +5. Otherwise the origin is unknown (a lock written before origins were + recorded, or none) or is not a listed baseline, and the engine **infers** a + merge base. It orders the listed baselines by how many frontmatter leaf + values differ from the live type's (a key present on one side only counts + once; equal-length arrays compare item by item; any other differing value + counts once), fewest first and in listed order on a tie. It takes the first + baseline whose three-way merge succeeds and whose merged type validates and + implements every exact contract version the desired type does, judged over + the pack's own contract resources: `update` with that merge, or `preserve` + when the merge changes nothing. +6. Otherwise, when the desired type declares `implements` and the live type + already validates and implements every exact contract version the desired + type does: `preserve`. The seed already fits the pack. +7. Otherwise: `preserve`, with a `reason` stating that no upgrade baseline + applies. The type is left as it is; it does not make the pack conflict. + +In cases 1, 2, 3, and 5 the lock's origin becomes the desired document; in the +others it is carried forward. An engine MUST NOT choose a merge baseline any +other way. An inferred base (case 5) may not be the seed's true origin, so the +differences between them could be applied as if they were the user's edits; the +conservative merge bounds this, since a seed whose `version` or other setting +differs from both sides is a competing change and fails. The assessment +reports, for every seed `update`, the baseline used as +`upgrade_baseline: { digest, version? }`, so the change is reviewed like any +other. + +A three-way merge is conservative. The type kind and name MUST match. Unchanged publisher +settings retain live customizations; unchanged live settings accept publisher +changes. Object settings merge recursively. Contract implementations merge by +contract ID only when unambiguous, retaining customized field mappings and +bindings while updating exact version references. Competing changes (including +ambiguous implementation lists) MUST fail closed. Other arrays merge item by +item when the baseline, live, and desired arrays have the same length, and are +otherwise indivisible. +Missing values and explicit null are distinct. Top-level setting removal +requires separate manual review. The live Markdown body and unrelated YAML +nodes MUST be preserved. Reformatting a changed YAML node is permitted. + +The merged resource digest MUST appear in the assessment. Normal assessment +and collection revision guards still apply. The upgrade, contract resources, +and provenance MUST publish atomically only after validating the staged +collection. A remaining reference to a removed exact contract MUST block the +whole upgrade; implementations MUST NOT rewrite additional types implicitly. +Record documents MUST NOT be migrated by this mechanism. An intentionally +preserved seed target MUST NOT be upgraded. + +This is not version negotiation or grant migration. Callers coordinating +applications must retain old contracts and implementations when supported +applications still require them, or arrange a separately reviewed coordinated +upgrade. Existing grants never acquire the new contract version implicitly. + +Resource digests are SHA-256 over the exact resource bytes. Source and target +paths are relative, forward-slash paths without traversal. +A directory, archive, repository, or package that distributes a pack MUST +preserve those bytes exactly, including line endings. Rewriting a text resource +from LF to CRLF therefore creates a different resource and MUST fail digest +verification. Distributors that store packs in version control SHOULD disable +line-ending conversion for pack resources. + +Type packs are installation units, not record types and not permission grants. +A pack may include several contracts, several implementing or auxiliary types, +and local JSON Schemas referenced by those artifacts. + +## Pack Identity And Portable Provenance + +The pack digest is SHA-256 over RFC 8785 JSON Canonicalization Scheme bytes for +the complete validated manifest, prefixed with `sha256:`. Because every +resource digest is part of the manifest, the pack digest identifies the exact +manifest and exact resource bytes without incorporating their distribution +location. + +Managed pack state is stored in `mdbase.lock.yaml` at the collection root and +validates against `schemas/v0.3/type-pack-lock.schema.json`. The lock is +portable collection source, not derived cache state. It records each installed +pack's exact ID, version, pack digest, stable installer identity, and the kind, +mode, canonical source, resolved target, and installed digest of every resource. Tools write it +deterministically and users MAY inspect or version it. Tools MUST NOT infer +ownership from filenames, application names, or `x-*` metadata. + +A seed resource's entry also records `origin_digest`, the digest of the +publisher document its target descends from, when that is known. Because a seed +becomes user-owned, the installed digest of a seed describes the pack, not the +target, and MUST NOT be used as its origin. An apply sets `origin_digest` to the +desired resource digest when it creates the seed, when it upgrades it (by exact +replacement or by merge), and whenever the target's live bytes equal the desired +document, whatever else applies. Otherwise it carries the previous entry's +`origin_digest` for that target forward unchanged, or omits it when there is +none. A seed target that existed before the pack was installed, and an +intentionally preserved seed target, therefore have no origin unless their bytes +equal the desired document. A lock entry without `origin_digest` means the +origin is unknown. Locks written before `origin_digest` existed carry none, so +an edited seed under such a lock is preserved with a reason rather than merged +until it is upgraded or recreated; an unedited seed still upgrades, because its +bytes prove its origin. `origin_digest` is defined only for seed resources and +MUST NOT appear on a managed resource. An apply that changes only seed origins leaves the pack's +status `current`; it is not a reconfiguration. + +Full collection snapshots, authority transfers, and unscoped synchronization +MUST carry `mdbase.lock.yaml` when it exists. A scoped application projection +MAY omit it to avoid disclosing unrelated pack metadata. A storage provider +MUST NOT reject the lock under a per-document limit that the collection owner +cannot remediate, because tools generate it. + +An absent lock means no resource is pack-managed, including resources created +by older one-shot installers. Adopting existing files into managed ownership is +a separate explicit operation and MUST verify their exact digests. + +## Assessment And Transactional Apply + +Pack evolution has two public phases: `assess_type_pack` is read-only and +`apply_type_pack` consumes a reviewed assessment. A caller supplies the desired +manifest, exact resources, and a stable reverse-domain `installed_by` +application or tool identity. Optional `target_overrides` map canonical manifest +targets to collection-specific paths without changing the publisher-owned pack +identity. Overrides are validated as safe collection paths, are included in the +assessment digest, and the resolved targets are written to the receipt. + +Assessment reports one of `current`, `install`, `upgrade`, `downgrade`, +`reconfigure`, or `conflict`, the current and desired pack identities, and every resource in a +stable exact diff. Resource actions are `create`, `update`, `delete`, +`adopt`, `unchanged`, `preserve`, or `conflict`. `adopt` records an existing +byte-identical managed resource without rewriting it. `preserve` is used for seed resources +and for seed resources retired from a newer pack. A conflicting assessment is +not applicable. The assessment also reports the planned `mdbase.lock.yaml` +action (`create`, `update`, or `unchanged`) and its resulting digest, so storage +adapters can persist the complete authority transaction without inferring +hidden engine writes. + +`reconfigure` means the exact same immutable pack is being resolved to different +collection targets. Managed resources are relocated only when their installed +bytes are unchanged; modified resources conflict. Seed resources are never +moved or deleted automatically. A caller may name seed targets in +`preserve_seed_targets` to record an intentional omission, such as when a user +maps the contract to an existing type instead of accepting the starter. Unknown +or non-seed preservation targets MUST be rejected. + +When guided setup maps a provided contract to an existing user-owned type, the +reviewed `contract_setups` choices are part of the same assessment and apply. +The assessment reports their exact type-resource diff and binds the selected +type revisions, field mappings, and bindings into `assessment_digest`. Apply +MUST stage those edits together with the managed resources and lock, validate +the combined collection, and commit them in the same transaction. A stale type +revision therefore leaves both the user-owned type and every pack resource +unchanged. A connector MUST NOT install the pack first and apply the reviewed +mapping as a later independent mutation. + +The assessment includes a deterministic `assessment_digest` over the desired +pack identity, current lock entry, and every relevant live target digest. Apply +MUST recompute it inside the collection mutation boundary and fail with +`concurrent_modification` before writing when it differs. Downgrades require an +explicit `allow_downgrade` decision. + +An unmanaged target with different bytes is a conflict unless the assessment +request contains an explicit adoption decision naming that target and its exact +current digest. An accepted adoption may plan `update`, records the adopted +digest in the diff, and is covered by `assessment_digest`. This is the safe +upgrade bridge for collections created before pack receipts existed: tools can +show the files that will become package-managed, require informed approval, and +remain race-safe. Unknown targets, seed targets, and stale adoption digests MUST +be rejected. Adoption is never inferred from application access or install +intent. + +A pack-aware apply implementation MUST: + +1. validate the manifest, safe paths, source bytes, and resource digests +2. stage every resource and reviewed existing-type setup without changing the live collection +3. resolve the complete staged contract and type registries +4. validate every contract, implementation, type, reference, and affected + existing record +5. compute and present the exact resource diff and assessment digest +6. acquire its collection mutation boundary and recheck the assessment +7. commit all resource changes and the lock as one recoverable transaction +8. reopen the collection and verify the committed registry and lock + +An invalid resource aborts before any live write. A conflict or concurrent +change aborts with the live collection unchanged. Implementations may use an +atomic directory exchange or a durable backup journal with rollback. After a +crash, recovery MUST restore either the complete pre-install state or the +complete committed state before normal collection operations resume. + +Revoking an application's access does not uninstall its type pack. Uninstall is +a separate, explicitly requested operation because records may still depend on +the installed types. + +Applying a current pack is valid and reports every managed resource as +`unchanged`; it MUST NOT create a new logical collection revision. Applying an +upgrade deletes a retired managed target only when its live digest still +matches the lock. User-modified managed resources produce `conflict` and no +live write. Successful apply reports the committed receipt, exact resource +diff, and `cleanup_deferred` when transaction-journal cleanup must be retried. + +## Diagnostics + +Type-pack-aware tools use these codes: + +| Code | Meaning | +| --- | --- | +| `invalid_type_pack` | a pack manifest, lock, resource, path, or digest is invalid | +| `type_pack_conflict` | ownership, live bytes, or an untracked target prevents a safe apply | +| `type_pack_apply_failed` | transactional commit or recovery did not complete normally | + +Diagnostics use the canonical shape from +[Chapter 14](../14-conformance.md). + +## Conformance + +Implementations of profile `type_packs/0.1` MUST: + +- validate pack manifests, safe resource paths, and resource digests +- read and write `mdbase.lock.yaml` deterministically +- produce a read-only assessment with an exact resource diff and + `assessment_digest` +- honor managed and seed resource ownership, explicit adoption, and target + overrides +- recheck the assessment inside the mutation boundary and fail with + `concurrent_modification` when it changed +- commit resources, reviewed type setups, and the lock as one recoverable + transaction +- treat reapplying a current pack as a no-op + +The fixtures are in `tests/v0.3/type-packs/type-packs.yaml`. From 048d6967f11a7a5534c86d2311f172d2adf867ce Mon Sep 17 00:00:00 2001 From: callumalpass Date: Thu, 12 Nov 2026 07:03:35 +1100 Subject: [PATCH 3/7] Dissolve Chapter 12A into the concurrent-edits/0.1 companion profile Move detection and the three-way record merge move verbatim to concurrent-edits/0.1.md as profiles move_detection/0.1 and merge/0.1. Record identity moves to Chapter 03 and writer format fidelity to Chapter 12. Core Write no longer depends on the body merge: rebasing body_edits is a merge/0.1 requirement, and watch reports an externally moved file as a rename only under move_detection/0.1. --- 00-overview.md | 10 +- 01-concepts.md | 7 +- 03-records-and-frontmatter.md | 21 ++- 04-configuration.md | 3 +- 07-collection-semantics.md | 9 +- 09-lifecycle.md | 2 +- 12-operations.md | 75 ++++++++-- 13-migrations-and-compatibility.md | 13 +- 14-conformance.md | 52 +++---- README.md | 14 +- .../0.1.md | 131 ++++++++---------- docs/releases/0.3.0-rc.5.md | 5 + schemas/v0.3/conformance-claim.schema.json | 9 +- scripts/check_v03_tests.py | 3 +- scripts/concurrent_edits_model.py | 15 +- site/build.mjs | 3 +- tests/v0.3/README.md | 2 +- tests/v0.3/core/body-edits-update.yaml | 2 +- tests/v0.3/core/list-ops-and-fidelity.yaml | 4 +- .../valid-all-profile-dependencies.yml | 2 + tests/v0.3/manifest.yaml | 14 +- tests/v0.3/merge/body-edits.yaml | 20 +-- tests/v0.3/merge/merge.yaml | 88 ++++++------ tests/v0.3/watch/move-detection.yaml | 26 ++-- 24 files changed, 301 insertions(+), 229 deletions(-) rename 12a-concurrent-edits.md => concurrent-edits/0.1.md (78%) diff --git a/00-overview.md b/00-overview.md index 9e6c88d..4e1ddd1 100644 --- a/00-overview.md +++ b/00-overview.md @@ -270,7 +270,8 @@ explicitly opts into `enforce: write`. When two edits to one record meet, for example an application update and an edit made in a text editor, a tool that reconciles them uses the three-way -record merge of Chapter 12A. Different fields merge, timestamps take the +record merge of the +[concurrent edits profile](./concurrent-edits/0.1.md). Different fields merge, timestamps take the later value, set-like lists take the union, appends to the body are both kept, and only real disagreements are conflicts. How a tool surfaces a conflict, and whether it replicates collections at all, is outside this specification. @@ -310,7 +311,7 @@ keep those claims precise and independently testable. | --- | --- | | [01-concepts.md](./01-concepts.md) | Core terminology and data model | | [02-collection-layout.md](./02-collection-layout.md) | Collection discovery, paths, and reserved state | -| [03-records-and-frontmatter.md](./03-records-and-frontmatter.md) | Markdown parsing and YAML value semantics | +| [03-records-and-frontmatter.md](./03-records-and-frontmatter.md) | Markdown parsing, record identity, and YAML value semantics | | [04-configuration.md](./04-configuration.md) | The `mdbase.yaml` configuration file | | [05-type-files.md](./05-type-files.md) | JSON Schema type wrappers and metadata | | [05a-data-contracts.md](./05a-data-contracts.md) | versioned data contracts and type implementations | @@ -320,8 +321,7 @@ keep those claims precise and independently testable. | [09-lifecycle.md](./09-lifecycle.md) | Managed values and mutation-time policy | | [10-cel-profile.md](./10-cel-profile.md) | Portable expressions and host bindings | | [11-querying.md](./11-querying.md) | Filters, ordering, projection, and result envelopes | -| [12-operations.md](./12-operations.md) | Read and write operations, concurrency, and diagnostics | -| [12a-concurrent-edits.md](./12a-concurrent-edits.md) | Record identity, move detection, three-way merge, and writer format fidelity | +| [12-operations.md](./12-operations.md) | Read and write operations, writer format fidelity, concurrency, and diagnostics | | [13-migrations-and-compatibility.md](./13-migrations-and-compatibility.md) | Migration from earlier versions and compatibility | | [14-conformance.md](./14-conformance.md) | Profiles, claims, fixtures, and runners | @@ -330,6 +330,7 @@ chapters: | Document | Description | | --- | --- | +| [concurrent-edits/0.1.md](./concurrent-edits/0.1.md) | Concurrent edits profile 0.1: move detection and the three-way record merge | | [type-packs/0.1.md](./type-packs/0.1.md) | Type packs profile 0.1: pack manifests, the lock file, assessment, and transactional apply | | [interop/0.1.md](./interop/0.1.md) | Event and action interoperability profile 0.1 | | [runtime/0.2.md](./runtime/0.2.md) | Durable runtime companion profile 0.2: standard pack, authorization, admission, execution, and recovery | @@ -354,6 +355,7 @@ Companion profiles, artifacts, and schemas carry their own versions: | Versioned thing | Declared by | Current | Format | | --- | --- | --- | --- | | collection specification | `spec_version` in `mdbase.yaml` | `0.3.0` | semantic version | +| concurrent edits profile | profile IDs `merge/0.1` and `move_detection/0.1` | `0.1` | major.minor | | type packs profile | profile ID `type_packs/0.1` | `0.1` | major.minor | | event/action interoperability profile | conformance claim `interop_profile_version` | `0.1` | major.minor | | durable runtime profile | conformance claim `runtime_profile_version` | `0.2` | major.minor | diff --git a/01-concepts.md b/01-concepts.md index dead79e..6f3e4b7 100644 --- a/01-concepts.md +++ b/01-concepts.md @@ -25,8 +25,8 @@ may additionally include `collection.read_defaults`. A record is identified by its path. A record carries no required mdbase metadata: no ID, revision, or merge state is ever required in the file. Implementations MAY keep internal record identities outside the collection's -records and follow a record across moves with the move detection of Chapter -12A. +records (Chapter 03) and follow a record across moves with the move +detection of the [concurrent edits profile](./concurrent-edits/0.1.md). ## Type @@ -110,7 +110,8 @@ from type policy during the active mutation. A merge combines two concurrent edits of one record against their common base version. Each top-level frontmatter field has a merge strategy, declared in `collection.merge` or derived from the type, and the body merges line by line. -Chapter 12A defines the merge function. When and where merges happen is an +The [concurrent edits profile](./concurrent-edits/0.1.md) +defines the merge function. When and where merges happen is an implementation concern. ## Expression diff --git a/03-records-and-frontmatter.md b/03-records-and-frontmatter.md index a099bff..a833988 100644 --- a/03-records-and-frontmatter.md +++ b/03-records-and-frontmatter.md @@ -76,6 +76,25 @@ application owns, such as Obsidian `.base` files (see the [Obsidian Bases adapter](./adapters/obsidian-bases.md)), without a second storage, discovery, or authorization model. +## Record Identity + +A record is identified by its collection-relative path. Record files carry no +required mdbase metadata: + +- An implementation MAY keep an internal identity for each record, for example + to follow it across moves or to key its history. +- Such identities, revisions, merge bases, and other bookkeeping MUST be kept + outside record files, for example in `.mdbase/` or the implementation's own + storage. +- An implementation MUST NOT write an identity into a record's frontmatter or + body, and MUST NOT require one to read, validate, query, or write a record. + +A collection may still store its own identifiers in frontmatter, such as a +TaskNotes `id` field. When `settings.id_field` names such a field, it is used +for ID-based link resolution (Chapter 08) and as an identity hint for the move +detection of the [concurrent edits profile](./concurrent-edits/0.1.md). It +remains ordinary user data. + ## Frontmatter Value Frontmatter MUST parse to a YAML mapping. Empty frontmatter is an empty mapping. @@ -199,7 +218,7 @@ update that supplies a non-empty body for a YAML document record fails with (Chapter 12) is written exactly as supplied. A write that changes some frontmatter keys of an existing record follows the -format fidelity rule of Chapter 12A: it re-emits only the changed top-level +writer format fidelity rule of Chapter 12: it re-emits only the changed top-level entries, keeps every other entry byte-identical, including comments, quoting, blank lines, and order, and keeps a changed entry's collection style. The rule applies to Markdown records and YAML document records alike. diff --git a/04-configuration.md b/04-configuration.md index d4980b0..e7fd08c 100644 --- a/04-configuration.md +++ b/04-configuration.md @@ -106,7 +106,8 @@ validation level, and it is not reported when the record is read later. Writes that do not go through an engine's write operations are never rejected. That includes edits made by other tools, files that appear through a file -system or synchronization tool, and the results of the merge in Chapter 12A. +system or synchronization tool, and the results of a merge +([concurrent edits profile](./concurrent-edits/0.1.md)). Their issues are reported when the record is read or validated. ## Validation Levels diff --git a/07-collection-semantics.md b/07-collection-semantics.md index 267b7b9..31f05c3 100644 --- a/07-collection-semantics.md +++ b/07-collection-semantics.md @@ -311,7 +311,7 @@ SHOULD generate them with lifecycle `ulid` or `uuid`. An `enforce: write` rule rejects only writes that introduce a duplicate. A write that leaves a record's covered values unchanged succeeds even when the record already violates the rule, for example after an external edit. Edits -made by other tools and the results of a merge (Chapter 12A) are never +made by other tools and the results of a merge are never rejected; their violations are reported. **Provisional (rc.5).** An `enforce: write` rule applies at every @@ -526,7 +526,7 @@ rules. In summary: The collision rule of Chapter 02 still applies to paths no single write chose: records created concurrently, records copied onto another file system, -and merged paths (Chapter 12A). Its suffix then becomes part of the decoded +and merged paths. Its suffix then becomes part of the decoded value, such as `Call Bob (2)`. **Provisional (rc.6).** Only string fields can be path-stored, and FORMAT @@ -537,7 +537,7 @@ before runs of spaces are collapsed and before the ends are trimmed. ## Merge Strategies `collection.merge` declares how each top-level frontmatter field combines when -two concurrent edits of a record are merged (Chapter 12A): +two concurrent edits of a record are merged: ```yaml collection: @@ -558,7 +558,8 @@ strategy: | `min` | the lesser of the two values | | `union` | an observed-remove set union of the two lists | -Chapter 12A defines each strategy exactly. A strategy matters only when both +The [concurrent edits profile](./concurrent-edits/0.1.md) +defines each strategy exactly. A strategy matters only when both sides changed the same field differently. A field changed on one side always takes that side's value, and a field changed identically on both sides takes the shared value. diff --git a/09-lifecycle.md b/09-lifecycle.md index 05ad39a..59dd92b 100644 --- a/09-lifecycle.md +++ b/09-lifecycle.md @@ -125,7 +125,7 @@ or `{ today: true }` merges with the `max` strategy by default (Chapter 07), so two concurrent updates that both refresh `dateModified` keep the later value instead of conflicting. A merge does not run lifecycle: lifecycle values in a merged record come from the two sides through their merge strategies -(Chapter 12A). +([concurrent edits profile](./concurrent-edits/0.1.md)). ## Place In The Write Order diff --git a/12-operations.md b/12-operations.md index 456a1ce..8c78636 100644 --- a/12-operations.md +++ b/12-operations.md @@ -161,7 +161,8 @@ Pipeline: type membership did not change as a lifecycle side effect or by the move 6. validate JSON Schema 7. run collection validators, applying the validation tiers of Chapter 04 -8. write the changed frontmatter entries with format fidelity (Chapter 12A) +8. write the changed frontmatter entries with format fidelity (Writer Format + Fidelity, below) and preserve the body 9. update derived indexes 10. emit watch/runtime events @@ -291,16 +292,20 @@ The update applies the edits as follows: 1. If the record's current body has the digest `body_base`, the edits are applied to it directly. -2. Otherwise the implementation obtains the base body, from `body_base_text` - or from bodies it has retained, and rebases the edits onto the current - body with the three-way body merge of Chapter 12A, where the current body - is the first version and the base with the edits applied is the second. - A merged body is written. A body conflict fails the update with - `concurrent_modification`, `details.reason: body_conflict`, and the +2. Otherwise an implementation that claims the `merge/0.1` profile obtains + the base body, from `body_base_text` or from bodies it has retained, and + rebases the edits onto the current body with the three-way body merge of + the [concurrent edits profile](./concurrent-edits/0.1.md), where the + current body is the first version and the base with the edits applied is + the second. A merged body is written. A body conflict fails the update + with `concurrent_modification`, `details.reason: body_conflict`, and the conflict; nothing is written. 3. If the base body cannot be obtained, the update fails with `concurrent_modification` and `details.reason: body_base_unavailable`. The caller can retry with `body_base_text` or against the current body. + An implementation that does not claim `merge/0.1` does not rebase: it + treats every base body as unavailable, whether or not the request + carries `body_base_text`. A request is `invalid_request` before any write when it combines `body_edits` with `body` or `document`, has `body_edits` without `body_base`, has an @@ -310,11 +315,11 @@ has no body. Body edits combine with `patch`, `unset`, `add`, and `remove` in one update, and lifecycle runs once for the whole update. Body edits need no live-collaboration machinery. An implementation that only -ever applies them directly, case 1 above, conforms, as long as it reports the -other cases as specified. +ever applies them directly, case 1 above, conforms to Core Write, as long as +it reports every other case as `body_base_unavailable`. Conflicts are found at line granularity, because they come from the -line-based body merge of Chapter 12A: two edits to different words of one line +line-based body merge of the concurrent edits profile: two edits to different words of one line conflict. A later release may detect conflicts at a finer granularity; that is a compatible refinement, because it only turns some conflicts into merges. @@ -380,8 +385,8 @@ referring record's `path`, the `field` that held the link (or `location: body`), `old_value`, and `new_value`, ordered by referring path. A rename keeps the record's identity: internal identity, history, and merge -bases follow the record to its new path, as for a move detected under Chapter -12A, and a tool that reports watch notifications reports it as +bases follow the record to its new path, as for a move detected under the +[concurrent edits profile](./concurrent-edits/0.1.md), and a tool that reports watch notifications reports it as `record_renamed`. An implementation MAY commit a rename and its reference updates as one atomic @@ -559,6 +564,45 @@ Update and delete use the concurrency behavior defined below. A successful create or update returns the same fields as `read_view_source`; a successful delete returns `path` and `deleted: true`. +## Writer Format Fidelity + +Every tool that writes changes to the frontmatter of an existing record, +through an update, a lifecycle assignment, a merge, or a reference update +after a rename, MUST follow this rule. It applies to Markdown records and YAML +document records alike. + +A frontmatter source consists of **top-level entries** and the lines between +them. An entry is a line that begins a top-level key at column 0, together +with every following line up to the last line of that key's value, and then +any directly following blank and indented comment lines up to the last +indented comment line. Lines inside the value belong to the entry even when +they are blank or column-0 comments, such as a comment between two `- ` items +of a block sequence at column 0. Blank lines and comment lines at column 0 +after the value are not part of any entry. + +1. An entry whose key the write does not change MUST stay byte-identical, + including its comments, quoting, indentation, and position. +2. Lines that are not part of any entry MUST be kept, except that removing an + entry removes its own lines only. +3. A changed entry is re-emitted in place. When the previous value was a flow + collection, such as `tags: [a, b]`, the new value MUST be written as a flow + collection, and a block collection MUST stay in block style. Writers + SHOULD keep a scalar's quoting style and a trailing comment on the entry's + first line when the new value can be written that way. +4. A new key is appended after the last entry. +5. A value that a merge takes unchanged from one side MUST be copied verbatim + from that side's source text for the entry. +6. The byte-order mark, the frontmatter delimiters, the line ending style, and + the body, unless the write changes the body, MUST stay as they were. + +A whole-document `document` replacement is written exactly as supplied and +is outside this rule. + +The rule keeps unrelated bytes stable, so files diff cleanly in Git and a +write never manufactures a change on a line it did not mean to touch. Byte +changes beyond the edited keys would also turn into needless conflicts in the +next merge. + ## Concurrency Write-capable tools SHOULD detect external modification between read and write @@ -578,7 +622,8 @@ A write without `if_revision` describes its own change: the keys it sets or removes, its list operations, and its body edit. When the record changed after the caller read it, an implementation either applies that change to the current record, as the update pipeline above does, or reconciles the two -versions with the three-way record merge of Chapter 12A. It MUST NOT discard +versions with the three-way record merge of the +[concurrent edits profile](./concurrent-edits/0.1.md). It MUST NOT discard the other change silently. A conflict found by the merge is reported; how it is held, shown, and resolved is implementation behavior. @@ -617,8 +662,8 @@ runtime state, or revisions. ## Events After a successful mutation, tools MAY emit watch/runtime events. A tool that -reports moves of files edited outside it uses the move detection of Chapter -12A. Events MUST be +reports moves of files edited outside it uses the move detection of the +[concurrent edits profile](./concurrent-edits/0.1.md). Events MUST be delivered after the derived read/query state is consistent. A change of path that changes path-stored values, whether by an update, a diff --git a/13-migrations-and-compatibility.md b/13-migrations-and-compatibility.md index 89828ca..6b7f02b 100644 --- a/13-migrations-and-compatibility.md +++ b/13-migrations-and-compatibility.md @@ -11,7 +11,9 @@ Migration never rewrites a record. ## Changes Since rc.4 -0.3.0-rc.5 adds the concurrent-edit semantics of Chapter 12A and relaxes +0.3.0-rc.5 adds concurrent-edit semantics (record identity in Chapter 03, +writer format fidelity in Chapter 12, and the +[concurrent edits profile](./concurrent-edits/0.1.md)) and relaxes several write-time checks. Collections keep `spec_version: "0.3.0"` and need no edit. Record files, type files, contracts, type packs, view records, and queries that were valid under rc.4 stay valid. @@ -29,7 +31,7 @@ worked under rc.4 stops loading. | `unique.scope` | listed, but its comparison set was underspecified | a rule governs records of its type; `scope` selects the records they must differ from; raw values compare without coercion; the default scope is `type` | nothing; check reports for records in the now exact scope | | link `validate_exists` and `target_type` | at level `error`, a write with a broken link fails | reported, never blocking | no equivalent; cross-record checks never block | | a successful write with cross-record issues | not applicable | returns `valid: true` with `warning` diagnostics | nothing | -| concurrent edits | whole-record revision checks | tools that reconcile edits merge field by field (Chapter 12A); `if_revision` is opt-in and never added on a caller's behalf | pass `if_revision` explicitly | +| concurrent edits | whole-record revision checks | tools that reconcile edits merge field by field (concurrent edits profile); `if_revision` is opt-in and never added on a caller's behalf | pass `if_revision` explicitly | | lifecycle `now` and `today` fields, `tags`, `uniqueItems` arrays | no merge semantics | merge as `max`, `union`, and `union` by default | declare `conflict` in `collection.merge` | | update | `patch`, `unset`, `body`, `document` | adds `add` and `remove` list operations, and `body_edits` against a `body_base` digest | nothing | | regular expressions in CEL `matches()`, JSON Schema `pattern`, and `match.where` | engine-dependent; Unicode-aware classes were implied | one profile everywhere: RE2 syntax with ASCII-only `\d`, `\w`, `\s`, `\b`, and case folding (Chapter 10); `\p{...}` is invalid | none; list non-ASCII characters explicitly, or lowercase text before matching | @@ -74,9 +76,10 @@ An engine moving from rc.4 to rc.5: - implements `add` and `remove`, path keys and the collision rule, `path_value_invalid`, and the writer format fidelity rule - stops supplying `if_revision` on a caller's behalf -- implements the merge of Chapter 12A if it reconciles concurrent edits, and - claims the `merge` profile -- implements move detection if it reports renames of externally moved files +- implements the merge of the [concurrent edits profile](./concurrent-edits/0.1.md) + if it reconciles concurrent edits, and claims the `merge/0.1` profile +- implements move detection, and claims the `move_detection/0.1` profile, if + it reports renames of externally moved files - reports `nondeterministic_match`, `path_collision`, and `ambiguous_link` - applies the filename tiebreakers exactly, and rewrites on rename only the links that resolved to the renamed record diff --git a/14-conformance.md b/14-conformance.md index 741a9cb..d682878 100644 --- a/14-conformance.md +++ b/14-conformance.md @@ -16,7 +16,6 @@ queries, writes, runtime preflight, workflow execution, and watching. | Query | evaluate contextual CEL filters, projections, grouping, summaries, and query envelopes | | Links | parse, resolve, validate, and traverse links | | Core Write | create, update, delete, rename, and batch records | -| Merge | merge concurrent versions of a record with declared field strategies | | Lifecycle | apply standard managed-field policy during writes | | Watch | report ordered collection changes after consistent state | @@ -24,6 +23,8 @@ Companion profiles add optional behavior on top of these: | Companion profile | Purpose | | --- | --- | +| Move Detection 0.1 | recognize a file moved outside the implementation as the same record | +| Merge 0.1 | merge concurrent versions of a record with declared field strategies | | Type Packs 0.1 | assess and transactionally apply managed type packs | | Event/Action Interoperability 0.1 | exchange CloudEvents and admitted action invocations through independently claimable roles | | Durable Runtime 0.2 | validate standard runtime records, admit exact plans through interoperability declarations, and execute/recover them durably | @@ -40,7 +41,6 @@ Normative profile IDs and dependencies of this specification are: | `cel_query` | `collection_semantics`, `cel` | | `links` | `collection_semantics`, `cel` | | `core_write` | `collection_semantics` | -| `merge` | `core_write` | | `lifecycle` | `core_write`, `cel` | | `watch` | `collection_semantics` | @@ -49,6 +49,8 @@ their profile IDs carry their version: | Profile ID | Requires | Defined by | | --- | --- | --- | +| `move_detection/0.1` | `watch` | [concurrent edits profile](./concurrent-edits/0.1.md) | +| `merge/0.1` | `core_write` | [concurrent edits profile](./concurrent-edits/0.1.md) | | `type_packs/0.1` | `data_contracts`, `core_write` | [type packs profile](./type-packs/0.1.md) | | `event_action_interop/0.1` | none | [event/action interoperability profile](./interop/0.1.md) | | `runtime/0.2` | `data_contracts`, `event_action_interop/0.1`, `cel` | [durable runtime profile](./runtime/0.2.md) | @@ -84,7 +86,8 @@ or specification version and SHOULD report stale evidence. ### Conformance Suites The shared fixtures live in `tests/v0.3/`, and `tests/v0.3/README.md` defines -their format. That includes the `merge_records` form, which states a merge case +their format. That includes the `merge_records` form of the `merge/0.1` +profile, which states a merge case as a base version, two edited versions, and the type declarations, with the exact merged bytes and conflicts as the expected result. Each test added in a release candidate after rc.4 records that release candidate, such as @@ -373,11 +376,11 @@ Core Write implementations MUST: - validate a complete draft before writing, rejecting only request and safety failures and, at level `error`, single-record issues - preserve unrelated Markdown body content -- follow the writer format fidelity rule of Chapter 12A +- follow the writer format fidelity rule of Chapter 12 - apply `add` and `remove` list operations to the current value -- apply `body_edits` directly when the body matches `body_base`, rebase them - with the Chapter 12A body merge when the base body is available, and report - `body_conflict` and `body_base_unavailable` otherwise +- apply `body_edits` directly when the body matches `body_base`, and + otherwise report `body_base_unavailable` unless they rebase them under the + `merge/0.1` profile - reject paths that escape the collection root - when a rename updates references, rewrite the links that resolved to the renamed record and would no longer resolve to it, including @@ -401,24 +404,6 @@ Core Write implementations MUST: - return the canonical operation envelope and final record revision - update derived state before reporting a successful mutation -## Merge Requirements - -Merge implementations MUST: - -- merge frontmatter one top-level key at a time with the three-way rules of - Chapter 12A -- apply declared and default `conflict`, `max`, `min`, and `union` - strategies exactly -- treat end-of-body append-append as a union, earlier-ordered side first -- merge other body changes line by line and report overlapping changes as a - `body` conflict -- keep the first version's value at every conflict and report the conflict - with its base, first, and second values -- never reject a merged version or turn it into a conflict because it is - invalid -- copy values taken unchanged from one side verbatim from that side's source - text - ## Lifecycle Requirements Lifecycle implementations MUST: @@ -433,6 +418,11 @@ Lifecycle implementations MUST: `type_membership_changed` - report conflicts between matched lifecycle policies +## Concurrent Edits Requirements + +The requirements of the `move_detection/0.1` and `merge/0.1` profiles are +defined by the [concurrent edits profile](./concurrent-edits/0.1.md). + ## Type Packs Requirements The `type_packs/0.1` profile's requirements, and the diagnostic codes @@ -481,9 +471,13 @@ An implementation that also claims `data_contracts` reports `contract_changed` with the contract file `path` after the contract registry and type implementations have been re-resolved. Implementations MAY also report `schema_changed`, `view_changed`, and `lock_changed`, each with a `path`, for -referenced local schema files, saved-view sources, and `mdbase.lock.yaml`. A rename that move -detection (Chapter 12A) does not pair is reported as `record_deleted` followed -by `record_created`. +referenced local schema files, saved-view sources, and `mdbase.lock.yaml`. + +A file moved outside the implementation is reported as `record_renamed` only +by an implementation that claims `move_detection/0.1`, and only when the +move detection of the [concurrent edits profile](./concurrent-edits/0.1.md) +pairs it. Every other such move is reported as `record_deleted` followed by +`record_created`. `record_created` and `record_modified` include current effective frontmatter. `record_renamed` includes `path`, `previous_path`, and current effective @@ -506,5 +500,3 @@ Watch implementations MUST: - coalesce duplicate host notifications for one logical change and report the final observed state - isolate listener failures so later notifications continue to be delivered -- pair disappearances and appearances with the move detection of Chapter 12A - when reporting renames of files changed outside the implementation diff --git a/README.md b/README.md index 6353fc1..3f5de19 100644 --- a/README.md +++ b/README.md @@ -112,6 +112,8 @@ order_by: | Create a collection | [Collection Layout](./02-collection-layout.md), [Configuration](./04-configuration.md), [Type Files](./05-type-files.md), and [Data Contracts](./05a-data-contracts.md) | | Validate, query, or save views over records | [JSON Schema Profile](./06-json-schema-profile.md), [CEL Profile](./10-cel-profile.md), and [Querying](./11-querying.md) | | Add links or managed fields | [Links](./08-links.md) and [Lifecycle](./09-lifecycle.md) | +| Reconcile concurrent edits | [Concurrent Edits Profile](./concurrent-edits/0.1.md) | +| Install and upgrade types | [Type Packs Profile](./type-packs/0.1.md) | | Connect applications | [Event and Action Interoperability](./interop/0.1.md) | | Define durable automation | [Durable Runtime Companion Profile](./runtime/0.2.md) and the [standard runtime pack](./standard-packs/mdbase-runtime/0.2.0/) | | Migrate a v0.2 collection | [Migrations And Compatibility](./13-migrations-and-compatibility.md) | @@ -139,14 +141,18 @@ independently. The specification is organized by topic: -- Chapters 00–04 introduce the model, records, collection layout, and +- Chapters 00–04 introduce the model, collection layout, records, and configuration. - Chapters 05–09 and 05A define types, data contracts, schemas, collection semantics, links, and lifecycle policy. - Chapters 10–12 define CEL expressions, queries, and record operations. -- Chapters 13–14 define the optional durable runtime companion built on core - contracts and event/action interoperability. -- Chapters 15–16 cover migration, compatibility, and conformance. +- Chapters 13–14 cover migration, compatibility, and conformance. + +Optional behavior lives in companion profiles with their own versions: +[concurrent edits](./concurrent-edits/0.1.md) (move detection and the +three-way merge), [type packs](./type-packs/0.1.md), +[event and action interoperability](./interop/0.1.md), and the +[durable runtime](./runtime/0.2.md). The [v0.2.1 archive](./v0.2/README.md) preserves the earlier specification and implementer material. Legacy fixtures are documented in diff --git a/12a-concurrent-edits.md b/concurrent-edits/0.1.md similarity index 78% rename from 12a-concurrent-edits.md rename to concurrent-edits/0.1.md index 92746e8..49771ca 100644 --- a/12a-concurrent-edits.md +++ b/concurrent-edits/0.1.md @@ -1,44 +1,34 @@ -# 12A. Concurrent Edits +# mdbase Concurrent Edits Profile 0.1 -## Scope +## Purpose A collection is edited by many tools at once: an application writing through -an engine, a person in a text editor, a synchronization tool, an agent. This -chapter defines the data semantics that let those edits meet without losing -either one: - -- how a record keeps its identity without any ID in its file, and how a moved - file is recognized as the same record -- the three-way record merge, which combines two concurrent versions of a - record against their common base -- the format fidelity rule that every writer follows, so that a write changes - only the bytes it means to change - -This specification does not define when a tool merges, how edits travel -between devices, logs, sequencers, ordering services, conflict envelopes, how -a conflict is held or shown to a person, or what happens when one side deletes +an engine, a person in a text editor, a synchronization tool, an agent. The +collection specification already gives every tool what it needs for those +edits to meet: a record keeps its identity without any ID in its file +(Chapter 03), types declare how each field combines (`collection.merge`, +Chapter 07), and every writer changes only the bytes it means to change (the +writer format fidelity rule, Chapter 12). + +This optional profile defines the two functions that a tool reconciling +concurrent edits adds to that: + +- **move detection**, which recognizes a moved file as the same record. + Profile identifier `move_detection/0.1`; it requires the `watch` profile. +- the **three-way record merge**, which combines two concurrent versions of + a record against their common base. Profile identifier `merge/0.1`; it + requires the `core_write` profile. + +The two are claimed independently. The profile is versioned independently of +the mdbase collection specification `0.3.0`, and chapter numbers in this +document refer to that specification. + +This profile does not define when a tool merges, how edits travel between +devices, logs, sequencers, ordering services, conflict envelopes, how a +conflict is held or shown to a person, or what happens when one side deletes a record that the other side edited. Those are implementation concerns. A tool that never reconciles concurrent edits, such as a command-line validator, does -not need this chapter's merge function and does not claim the `merge` -profile. - -## Record Identity - -A record is identified by its collection-relative path. Record files carry no -required mdbase metadata: - -- An implementation MAY keep an internal identity for each record, for example - to follow it across moves or to key its history. -- Such identities, revisions, merge bases, and other bookkeeping MUST be kept - outside record files, for example in `.mdbase/` or the implementation's own - storage. -- An implementation MUST NOT write an identity into a record's frontmatter or - body, and MUST NOT require one to read, validate, query, or write a record. - -A collection may still store its own identifiers in frontmatter, such as a -TaskNotes `id` field. When `settings.id_field` names such a field, it is used -for ID-based link resolution (Chapter 08) and as an identity hint for move -detection below. It remains ordinary user data. +not need this profile. ## Move Detection @@ -205,7 +195,7 @@ added it again. When the result is empty and `F` or `S` is a missing key, the key is missing from the merged version; otherwise the result is a list. A key whose value came from one side unchanged takes that side's source text -under the format fidelity rule below. A value computed by `union`, or a `max` +under the writer format fidelity rule of Chapter 12. A value computed by `union`, or a `max` or `min` result written in a different form, is re-emitted. The merged frontmatter source starts from the first version's source. Comment @@ -315,41 +305,32 @@ Membership of the merged version is recomputed from its path and frontmatter and may differ from the base. The merge does not run lifecycle; managed values such as `dateModified` combine through their strategies. -## Writer Format Fidelity - -Every tool that writes changes to the frontmatter of an existing record, -through an update, a lifecycle assignment, a merge, or a reference update -after a rename, MUST follow this rule. It applies to Markdown records and YAML -document records alike. - -A frontmatter source consists of **top-level entries** and the lines between -them. An entry is a line that begins a top-level key at column 0, together -with every following line up to the last line of that key's value, and then -any directly following blank and indented comment lines up to the last -indented comment line. Lines inside the value belong to the entry even when -they are blank or column-0 comments, such as a comment between two `- ` items -of a block sequence at column 0. Blank lines and comment lines at column 0 -after the value are not part of any entry. - -1. An entry whose key the write does not change MUST stay byte-identical, - including its comments, quoting, indentation, and position. -2. Lines that are not part of any entry MUST be kept, except that removing an - entry removes its own lines only. -3. A changed entry is re-emitted in place. When the previous value was a flow - collection, such as `tags: [a, b]`, the new value MUST be written as a flow - collection, and a block collection MUST stay in block style. Writers - SHOULD keep a scalar's quoting style and a trailing comment on the entry's - first line when the new value can be written that way. -4. A new key is appended after the last entry. -5. A value that a merge takes unchanged from one side MUST be copied verbatim - from that side's source text for the entry. -6. The byte-order mark, the frontmatter delimiters, the line ending style, and - the body, unless the write changes the body, MUST stay as they were. - -A whole-document `document` replacement (Chapter 12) is written exactly as -supplied and is outside this rule. - -The rule keeps unrelated bytes stable, so files diff cleanly in Git and a -write never manufactures a change on a line it did not mean to touch. Byte -changes beyond the edited keys would also turn into needless conflicts in the -next merge. +## Conformance + +Implementations of profile `move_detection/0.1` MUST: + +- pair disappearances and appearances with the rules of Move Detection when + reporting renames of files changed outside the implementation +- report each pair as `record_renamed`, followed by `record_modified` when + the bytes also changed + +Implementations of profile `merge/0.1` MUST: + +- merge frontmatter one top-level key at a time with the three-way rules of + this profile +- apply declared and default `conflict`, `max`, `min`, and `union` + strategies exactly +- treat end-of-body append-append as a union, earlier-ordered side first +- merge other body changes line by line and report overlapping changes as a + `body` conflict +- keep the first version's value at every conflict and report the conflict + with its base, first, and second values +- never reject a merged version or turn it into a conflict because it is + invalid +- copy values taken unchanged from one side verbatim from that side's source + text +- rebase the `body_edits` of an update (Chapter 12) onto a changed body with + the body merge when the base body is available + +The fixtures are in `tests/v0.3/merge/` and +`tests/v0.3/watch/move-detection.yaml`. diff --git a/docs/releases/0.3.0-rc.5.md b/docs/releases/0.3.0-rc.5.md index 94c0413..23fc3e0 100644 --- a/docs/releases/0.3.0-rc.5.md +++ b/docs/releases/0.3.0-rc.5.md @@ -2,6 +2,11 @@ **Status:** draft on branch `spec/v0.3.0-rc.5-draft`; not tagged. +These notes describe rc.5 as it was written. In rc.6, Chapter 12A was +divided: record identity is in Chapter 03, writer format fidelity is in +Chapter 12, and move detection and the three-way merge are the +[concurrent edits profile](../../concurrent-edits/0.1.md). + The fifth release candidate of mdbase 0.3.0 specifies what happens when a collection is edited by several tools at once: an application writing through an engine, a person in a text editor, a synchronization tool, an agent on diff --git a/schemas/v0.3/conformance-claim.schema.json b/schemas/v0.3/conformance-claim.schema.json index 69cef6a..c30d585 100644 --- a/schemas/v0.3/conformance-claim.schema.json +++ b/schemas/v0.3/conformance-claim.schema.json @@ -204,9 +204,13 @@ "then": { "properties": { "profiles": { "contains": { "const": "collection_semantics" } } } } }, { - "if": { "properties": { "profiles": { "contains": { "const": "merge" } } } }, + "if": { "properties": { "profiles": { "contains": { "const": "merge/0.1" } } } }, "then": { "properties": { "profiles": { "contains": { "const": "core_write" } } } } }, + { + "if": { "properties": { "profiles": { "contains": { "const": "move_detection/0.1" } } } }, + "then": { "properties": { "profiles": { "contains": { "const": "watch" } } } } + }, { "if": { "properties": { "profiles": { "contains": { "const": "lifecycle" } } } }, "then": { @@ -261,7 +265,8 @@ "cel_query", "links", "core_write", - "merge", + "merge/0.1", + "move_detection/0.1", "type_packs/0.1", "lifecycle", "event_action_interop/0.1", diff --git a/scripts/check_v03_tests.py b/scripts/check_v03_tests.py index 28cb603..ca6f0db 100755 --- a/scripts/check_v03_tests.py +++ b/scripts/check_v03_tests.py @@ -78,7 +78,8 @@ "cel_query", "links", "core_write", - "merge", + "merge/0.1", + "move_detection/0.1", "type_packs/0.1", "lifecycle", "event_action_interop/0.1", diff --git a/scripts/concurrent_edits_model.py b/scripts/concurrent_edits_model.py index 06d6828..2cae020 100644 --- a/scripts/concurrent_edits_model.py +++ b/scripts/concurrent_edits_model.py @@ -1,5 +1,6 @@ #!/usr/bin/env python3 -"""Executable model of the pure concurrent-edit functions in Chapters 02, 07, and 12A. +"""Executable model of the pure functions in Chapters 02, 07, and 12 and in +the concurrent edits profile 0.1. This is not an mdbase implementation. It models the functions that the 0.3.0-rc.5 conformance fixtures can check without a collection engine: @@ -8,11 +9,13 @@ - path-pattern derivation and placeholder validation (Chapter 07) - path-stored field normalization and decoding (Chapter 07, since rc.6) - default and declared merge strategies (Chapter 07) -- the three-way record merge and writer format fidelity (Chapter 12A) -- move detection pairing (Chapter 12A) +- the three-way record merge (concurrent edits profile) and writer format + fidelity (Chapter 12) +- move detection pairing (concurrent edits profile) - the mdbase regex profile (Chapter 10), approximated with Python `re` in ASCII mode for the constructs the fixtures use -- body edits and their rebase onto a changed body (Chapters 12 and 12A) +- body edits and their rebase onto a changed body (Chapter 12 and the + concurrent edits profile) `scripts/check_v03_tests.py` runs the corresponding fixtures against it through `run_fixture`. @@ -750,7 +753,7 @@ def merge_body(base: str, first: str, second: str) -> str | None: def _lcs_pairs(a: list[str], b: list[str]) -> list[tuple[int, int]]: - """The Chapter 12A alignment: Myers' linear-space algorithm. + """The concurrent edits profile's alignment: Myers' linear-space algorithm. Common prefixes and suffixes are matched first; the rest splits at the middle snake found by `_bisect` and each part is aligned the same way. @@ -996,7 +999,7 @@ def _take(result: Document, side: Document, key: str, eol: str) -> None: def _append_entry(result: Document, entry: Entry) -> None: - """Insert a new entry after the last existing entry (Chapter 12A rule 4).""" + """Insert a new entry after the last existing entry (Chapter 12, writer format fidelity rule 4).""" last = max((i for i, item in enumerate(result.items) if isinstance(item, Entry)), default=-1) result.items.insert(last + 1, entry) diff --git a/site/build.mjs b/site/build.mjs index 47df560..0b2baaf 100644 --- a/site/build.mjs +++ b/site/build.mjs @@ -86,9 +86,9 @@ const SPEC_FILES = [ { file: '10-cel-profile.md', num: '10', title: 'CEL Profile', id: 'section-10' }, { file: '11-querying.md', num: '11', title: 'Querying', id: 'section-11' }, { file: '12-operations.md', num: '12', title: 'Operations', id: 'section-12' }, - { file: '12a-concurrent-edits.md', num: '12A', title: 'Concurrent Edits', id: 'section-12a' }, { file: '13-migrations-and-compatibility.md', num: '13', title: 'Migrations & Compatibility', id: 'section-13' }, { file: '14-conformance.md', num: '14', title: 'Conformance', id: 'section-14' }, + { file: 'concurrent-edits/0.1.md', num: 'C', title: 'Concurrent Edits', id: 'companion-concurrent-edits', group: 'Companion Profiles' }, { file: 'type-packs/0.1.md', num: 'P', title: 'Type Packs', id: 'companion-type-packs', group: 'Companion Profiles' }, { file: 'interop/0.1.md', num: 'I', title: 'Event & Action Interoperability', id: 'companion-interop', group: 'Companion Profiles' }, { file: 'runtime/0.2.md', num: 'R', title: 'Durable Runtime', id: 'companion-runtime', group: 'Companion Profiles' }, @@ -189,6 +189,7 @@ renderer.link = function ({ href, title, text }) { } const companions = { + 'concurrent-edits/0.1.md': '#companion-concurrent-edits', 'type-packs/0.1.md': '#companion-type-packs', 'interop/0.1.md': '#companion-interop', 'runtime/0.2.md': '#companion-runtime', diff --git a/tests/v0.3/README.md b/tests/v0.3/README.md index f70cc98..b6f7f43 100644 --- a/tests/v0.3/README.md +++ b/tests/v0.3/README.md @@ -18,7 +18,7 @@ The tests below are grouped into thirteen fixture sets: 10. `runtime_contracts` 11. `workflow_execution` 12. `merge` (since rc.5) -13. `watch` (since rc.5) +13. `move_detection` (since rc.5; named `watch` before rc.6) The suite covers JSON Schema artifacts, type wrappers, first-class data contracts and projections, collection semantics, CEL host bindings, saved diff --git a/tests/v0.3/core/body-edits-update.yaml b/tests/v0.3/core/body-edits-update.yaml index cf08fb3..219279c 100644 --- a/tests/v0.3/core/body-edits-update.yaml +++ b/tests/v0.3/core/body-edits-update.yaml @@ -69,7 +69,7 @@ groups: - name: body_base_text lets a stale base rebase id: core_write.body_edits_base_text since: "0.3.0-rc.5" - covers: [core_write.body_edits, merge.body_edit_rebase] + covers: [merge/0.1.body_edit_rebase] operation: update input: path: notes/meeting.md diff --git a/tests/v0.3/core/list-ops-and-fidelity.yaml b/tests/v0.3/core/list-ops-and-fidelity.yaml index f30cd9c..d8daadc 100644 --- a/tests/v0.3/core/list-ops-and-fidelity.yaml +++ b/tests/v0.3/core/list-ops-and-fidelity.yaml @@ -1,11 +1,11 @@ # Core Write additions in 0.3.0-rc.5: list operations (Chapter 12), writer format -# fidelity (Chapter 12A), and path collisions on create and rename (Chapters +# fidelity (Chapter 12), and path collisions on create and rename (Chapters # 02 and 12). expect.document is the exact post-write source. name: "core write additions" spec_version: "0.3.0" fixture_set: core_collection category: operations -spec_ref: "v0.3/02, v0.3/03, v0.3/12, v0.3/12A" +spec_ref: "v0.3/02, v0.3/03, v0.3/12" groups: - name: "list operations and format fidelity" diff --git a/tests/v0.3/fixtures/conformance/valid-all-profile-dependencies.yml b/tests/v0.3/fixtures/conformance/valid-all-profile-dependencies.yml index eee193e..aa104e7 100644 --- a/tests/v0.3/fixtures/conformance/valid-all-profile-dependencies.yml +++ b/tests/v0.3/fixtures/conformance/valid-all-profile-dependencies.yml @@ -16,6 +16,8 @@ profiles: - cel_query - links - core_write + - merge/0.1 + - move_detection/0.1 - type_packs/0.1 - lifecycle - event_action_interop/0.1 diff --git a/tests/v0.3/manifest.yaml b/tests/v0.3/manifest.yaml index 8a6c0b0..9f3778b 100644 --- a/tests/v0.3/manifest.yaml +++ b/tests/v0.3/manifest.yaml @@ -128,7 +128,7 @@ claim_profiles: - path_stored_writes # since 0.3.0-rc.6 - path_value_normalized # since 0.3.0-rc.6 - create_membership_path # since 0.3.0-rc.6 - - id: merge # since 0.3.0-rc.5 + - id: merge/0.1 # since 0.3.0-rc.5; a companion profile since 0.3.0-rc.6 status: coverage_complete requires: [core_write] requirements: @@ -211,7 +211,11 @@ claim_profiles: - per_subject_ordering - notification_coalescing - listener_isolation - - move_detection # since 0.3.0-rc.5 + - id: move_detection/0.1 # since 0.3.0-rc.6; move_detection/0.1.move_pairing in rc.5 + status: coverage_complete + requires: [watch] + requirements: + - move_pairing fixture_sets: - id: schema_artifacts @@ -286,12 +290,12 @@ fixture_sets: - workflow-execution/workflow-execution.yaml - id: merge description: Three-way record merge and merge declarations, including the mdbase-next prototype merge fixtures (since 0.3.0-rc.5). - coverage_targets: [merge, collection_semantics] + coverage_targets: [merge/0.1, collection_semantics] files: - merge/merge.yaml - merge/body-edits.yaml - - id: watch + - id: move_detection description: Move detection for externally moved files (since 0.3.0-rc.5). - coverage_targets: [watch] + coverage_targets: [move_detection/0.1] files: - watch/move-detection.yaml diff --git a/tests/v0.3/merge/body-edits.yaml b/tests/v0.3/merge/body-edits.yaml index 7824767..8304c5b 100644 --- a/tests/v0.3/merge/body-edits.yaml +++ b/tests/v0.3/merge/body-edits.yaml @@ -1,4 +1,4 @@ -# Body edits (Chapters 12 and 12A), since 0.3.0-rc.5. Each apply_body_edits +# Body edits (Chapter 12 and concurrent edits profile 0.1), since 0.3.0-rc.5. Each apply_body_edits # case gives the base body, the record's current body, and the edits, whose # offsets count Unicode scalar values of the base. base_available: false # models an engine that has neither body_base_text nor a retained copy of the @@ -7,14 +7,14 @@ name: body edits against a known base spec_version: "0.3.0" fixture_set: merge category: merge -spec_ref: "v0.3/12, v0.3/12A" +spec_ref: "v0.3/12, concurrent-edits/0.1" groups: - name: apply and rebase body edits tests: - name: edits apply directly when the body is the base id: body_edits.direct_replace since: "0.3.0-rc.5" - covers: [merge.body_edit_rebase] + covers: [merge/0.1.body_edit_rebase] operation: apply_body_edits input: base: | @@ -37,7 +37,7 @@ groups: - name: every offset refers to the base body id: body_edits.base_coordinates since: "0.3.0-rc.5" - covers: [merge.body_edit_rebase] + covers: [merge/0.1.body_edit_rebase] operation: apply_body_edits input: base: | @@ -65,7 +65,7 @@ groups: - name: "offsets count Unicode scalar values, not bytes or UTF-16 units" id: body_edits.scalar_offsets since: "0.3.0-rc.5" - covers: [merge.body_edit_rebase] + covers: [merge/0.1.body_edit_rebase] operation: apply_body_edits input: base: | @@ -85,7 +85,7 @@ groups: - name: an empty text deletes its range id: body_edits.deletion since: "0.3.0-rc.5" - covers: [merge.body_edit_rebase] + covers: [merge/0.1.body_edit_rebase] operation: apply_body_edits input: base: | @@ -107,7 +107,7 @@ groups: - name: edits rebase onto a concurrent change to other lines id: body_edits.rebase_disjoint_lines since: "0.3.0-rc.5" - covers: [merge.body_edit_rebase] + covers: [merge/0.1.body_edit_rebase] operation: apply_body_edits input: base: | @@ -130,7 +130,7 @@ groups: - name: "an append rebases onto a concurrent append, current text first" id: body_edits.rebase_append_append since: "0.3.0-rc.5" - covers: [merge.body_edit_rebase] + covers: [merge/0.1.body_edit_rebase] operation: apply_body_edits input: base: | @@ -157,7 +157,7 @@ groups: - name: an edit to a line that changed concurrently is a conflict id: body_edits.rebase_conflict_same_line since: "0.3.0-rc.5" - covers: [merge.body_edit_rebase] + covers: [merge/0.1.body_edit_rebase] operation: apply_body_edits input: base: | @@ -180,7 +180,7 @@ groups: - name: a changed body without the base text cannot be rebased id: body_edits.base_unavailable since: "0.3.0-rc.5" - covers: [merge.body_edit_rebase, core_write.body_edits] + covers: [merge/0.1.body_edit_rebase, core_write.body_edits] operation: apply_body_edits input: base: | diff --git a/tests/v0.3/merge/merge.yaml b/tests/v0.3/merge/merge.yaml index 2e4aafa..6b92263 100644 --- a/tests/v0.3/merge/merge.yaml +++ b/tests/v0.3/merge/merge.yaml @@ -1,4 +1,4 @@ -# Three-way record merge fixtures (Chapter 12A). +# Three-way record merge fixtures (concurrent edits profile 0.1). # # Each merge_records test supplies the base, first, and second versions of one # record. "first" is the earlier-ordered version; at a conflict the merged @@ -14,7 +14,7 @@ name: "three-way record merge" spec_version: "0.3.0" fixture_set: merge category: merge -spec_ref: "v0.3/07, v0.3/12A" +spec_ref: "v0.3/07, concurrent-edits/0.1" groups: - name: "prototype merge fixtures (TaskNotes-like task type)" setup: @@ -69,7 +69,7 @@ groups: - name: "Both append different lines at the end: union (theirs first), no conflict. (prototype fixture body-append-append)" id: merge.body_append_append since: 0.3.0-rc.5 - covers: [merge.append_append_union] + covers: [merge/0.1.append_append_union] operation: merge_records input: path: tasks/T.md @@ -122,7 +122,7 @@ groups: - name: "Different body lines edited: both kept. (prototype fixture body-disjoint-hunks)" id: merge.body_disjoint_hunks since: 0.3.0-rc.5 - covers: [merge.body_three_way] + covers: [merge/0.1.body_three_way] operation: merge_records input: path: tasks/T.md @@ -191,7 +191,7 @@ groups: - name: "Same line edited differently: conflict; theirs stays. (prototype fixture body-overlap)" id: merge.body_overlap since: 0.3.0-rc.5 - covers: [merge.body_three_way, merge.conflict_reporting] + covers: [merge/0.1.body_three_way, merge/0.1.conflict_reporting] operation: merge_records input: path: tasks/T.md @@ -249,7 +249,7 @@ groups: - name: "completedDate declared merge: max. (prototype fixture declared-max)" id: merge.declared_max since: 0.3.0-rc.5 - covers: [merge.declared_strategies, merge.max_min_semantics] + covers: [merge/0.1.declared_strategies, merge/0.1.max_min_semantics] operation: merge_records input: path: tasks/T.md @@ -310,7 +310,7 @@ groups: - name: "Different keys changed on each side: both kept. (prototype fixture disjoint-fields)" id: merge.disjoint_fields since: 0.3.0-rc.5 - covers: [merge.field_three_way] + covers: [merge/0.1.field_three_way] operation: merge_records input: path: tasks/T.md @@ -367,7 +367,7 @@ groups: - name: "A field change and a body change merge. (prototype fixture field-vs-body)" id: merge.field_vs_body since: 0.3.0-rc.5 - covers: [merge.field_three_way, merge.body_three_way] + covers: [merge/0.1.field_three_way, merge/0.1.body_three_way] operation: merge_records input: path: tasks/T.md @@ -424,7 +424,7 @@ groups: - name: "Untouched entries keep comments/quoting; the changed key is copied verbatim from the side that changed it. (prototype fixture format-preserved)" id: merge.format_preserved since: 0.3.0-rc.5 - covers: [merge.merge_format_fidelity] + covers: [merge/0.1.merge_format_fidelity] operation: merge_records input: path: tasks/T.md @@ -481,7 +481,7 @@ groups: - name: "dateModified changed on both sides: the later timestamp wins, no conflict. (prototype fixture lifecycle-now-max)" id: merge.lifecycle_now_max since: 0.3.0-rc.5 - covers: [merge.default_strategies, merge.max_min_semantics] + covers: [merge/0.1.default_strategies, merge/0.1.max_min_semantics] operation: merge_records input: path: tasks/T.md @@ -538,7 +538,7 @@ groups: - name: "Ours sets priority 9 (violates maximum 5) via an external edit: accepted, validity reported, not a conflict. (prototype fixture merge-invalid-is-reported)" id: merge.merge_invalid_is_reported since: 0.3.0-rc.5 - covers: [merge.invalid_merge_reported] + covers: [merge/0.1.invalid_merge_reported] operation: merge_records input: path: tasks/T.md @@ -595,7 +595,7 @@ groups: - name: "Default merge: conflict. The earlier-sequenced value (theirs) stays. (prototype fixture same-field-different-values)" id: merge.same_field_different_values since: 0.3.0-rc.5 - covers: [merge.field_three_way, merge.conflict_reporting] + covers: [merge/0.1.field_three_way, merge/0.1.conflict_reporting] operation: merge_records input: path: tasks/T.md @@ -654,7 +654,7 @@ groups: - name: "Both sides set the same value: no conflict. (prototype fixture same-field-same-value)" id: merge.same_field_same_value since: 0.3.0-rc.5 - covers: [merge.field_three_way] + covers: [merge/0.1.field_three_way] operation: merge_records input: path: tasks/T.md @@ -711,7 +711,7 @@ groups: - name: "Both add different tags: union. (prototype fixture tags-union-add-add)" id: merge.tags_union_add_add since: 0.3.0-rc.5 - covers: [merge.default_strategies, merge.union_semantics] + covers: [merge/0.1.default_strategies, merge/0.1.union_semantics] operation: merge_records input: path: tasks/T.md @@ -768,7 +768,7 @@ groups: - name: "One side removes b, the other adds c: remove and add both apply. (prototype fixture tags-union-remove-add)" id: merge.tags_union_remove_add since: 0.3.0-rc.5 - covers: [merge.union_semantics] + covers: [merge/0.1.union_semantics] operation: merge_records input: path: tasks/T.md @@ -858,7 +858,7 @@ groups: - name: a field declared min keeps the lesser of two changed values id: merge.declared_min since: 0.3.0-rc.5 - covers: [merge.declared_strategies, merge.max_min_semantics] + covers: [merge/0.1.declared_strategies, merge/0.1.max_min_semantics] operation: merge_records input: path: items/a.md @@ -895,7 +895,7 @@ groups: - name: a declaration replaces the max default of a lifecycle now field id: merge.declaration_overrides_default since: 0.3.0-rc.5 - covers: [merge.declared_strategies, merge.conflict_reporting] + covers: [merge/0.1.declared_strategies, merge/0.1.conflict_reporting] operation: merge_records input: path: items/a.md @@ -934,7 +934,7 @@ groups: - name: "max compares date-times with offsets as instants, not as text" id: merge.max_compares_instants since: 0.3.0-rc.5 - covers: [merge.max_min_semantics] + covers: [merge/0.1.max_min_semantics] operation: merge_records input: path: items/a.md @@ -971,7 +971,7 @@ groups: - name: max of a number and a string is a conflict id: merge.max_incomparable_conflicts since: 0.3.0-rc.5 - covers: [merge.max_min_semantics, merge.conflict_reporting] + covers: [merge/0.1.max_min_semantics, merge/0.1.conflict_reporting] operation: merge_records input: path: items/a.md @@ -1006,7 +1006,7 @@ groups: - name: under max a value survives a concurrent removal id: merge.max_value_beats_removal since: 0.3.0-rc.5 - covers: [merge.max_min_semantics] + covers: [merge/0.1.max_min_semantics] operation: merge_records input: path: items/a.md @@ -1038,7 +1038,7 @@ groups: - name: an array with uniqueItems merges as a set by default and keeps block style id: merge.unique_items_default_union since: 0.3.0-rc.5 - covers: [merge.default_strategies, merge.union_semantics, merge.merge_format_fidelity] + covers: [merge/0.1.default_strategies, merge/0.1.union_semantics, merge/0.1.merge_format_fidelity] operation: merge_records input: path: items/a.md @@ -1081,7 +1081,7 @@ groups: - name: an array without uniqueItems has the conflict default id: merge.plain_array_conflicts since: 0.3.0-rc.5 - covers: [merge.default_strategies, merge.conflict_reporting] + covers: [merge/0.1.default_strategies, merge/0.1.conflict_reporting] operation: merge_records input: path: items/a.md @@ -1120,7 +1120,7 @@ groups: - name: union treats a removed key as an empty list and re-emits the result id: merge.union_with_removed_key since: 0.3.0-rc.5 - covers: [merge.union_semantics] + covers: [merge/0.1.union_semantics] operation: merge_records input: path: items/a.md @@ -1156,7 +1156,7 @@ groups: - name: an empty union result with a removed key leaves the key missing id: merge.union_empty_removes_key since: 0.3.0-rc.5 - covers: [merge.union_semantics] + covers: [merge/0.1.union_semantics] operation: merge_records input: path: items/a.md @@ -1191,7 +1191,7 @@ groups: - name: a tags string reads as a one-item list id: merge.tags_string_is_one_item since: 0.3.0-rc.5 - covers: [merge.union_semantics] + covers: [merge/0.1.union_semantics] operation: merge_records input: path: items/a.md @@ -1228,7 +1228,7 @@ groups: - name: a key removed on one side and unchanged on the other is removed id: merge.one_side_removes since: 0.3.0-rc.5 - covers: [merge.field_three_way] + covers: [merge/0.1.field_three_way] operation: merge_records input: path: items/a.md @@ -1263,7 +1263,7 @@ groups: - name: a key added on both sides with different values is a conflict id: merge.added_both_sides_differently since: 0.3.0-rc.5 - covers: [merge.field_three_way, merge.conflict_reporting] + covers: [merge/0.1.field_three_way, merge/0.1.conflict_reporting] operation: merge_records input: path: items/a.md @@ -1301,7 +1301,7 @@ groups: - name: setting null and removing a key are different changes id: merge.null_is_not_missing since: 0.3.0-rc.5 - covers: [merge.field_three_way, merge.conflict_reporting] + covers: [merge/0.1.field_three_way, merge/0.1.conflict_reporting] operation: merge_records input: path: items/a.md @@ -1339,7 +1339,7 @@ groups: - name: a key added by the second side goes after the last entry and before trailing comments id: merge.new_key_after_last_entry since: 0.3.0-rc.5 - covers: [merge.merge_format_fidelity] + covers: [merge/0.1.merge_format_fidelity] operation: merge_records input: path: items/a.md @@ -1378,7 +1378,7 @@ groups: - name: append-append inserts a line break after an append that lacks one id: merge.append_without_final_newline since: 0.3.0-rc.5 - covers: [merge.append_append_union] + covers: [merge/0.1.append_append_union] operation: merge_records input: path: items/a.md @@ -1415,7 +1415,7 @@ groups: - name: two insertions at the same place inside the body are a conflict id: merge.interior_insertions_conflict since: 0.3.0-rc.5 - covers: [merge.body_three_way, merge.conflict_reporting] + covers: [merge/0.1.body_three_way, merge/0.1.conflict_reporting] operation: merge_records input: path: items/a.md @@ -1456,7 +1456,7 @@ groups: - name: "when the first version is unchanged the merged version is the second, byte for byte" id: merge.unchanged_first_takes_second_bytes since: 0.3.0-rc.5 - covers: [merge.field_three_way, merge.merge_format_fidelity] + covers: [merge/0.1.field_three_way, merge/0.1.merge_format_fidelity] operation: merge_records input: path: items/a.md @@ -1491,7 +1491,7 @@ groups: - name: two identical edits merge to that edit id: merge.identical_edits since: 0.3.0-rc.5 - covers: [merge.field_three_way, merge.body_three_way] + covers: [merge/0.1.field_three_way, merge/0.1.body_three_way] operation: merge_records input: path: items/a.md @@ -1524,7 +1524,7 @@ groups: - name: a move on one side wins over an unchanged path id: merge.path_one_side_moves since: 0.3.0-rc.5 - covers: [merge.path_merge] + covers: [merge/0.1.path_merge] operation: merge_records input: paths: @@ -1561,7 +1561,7 @@ groups: - name: two different moves are a path conflict id: merge.path_two_moves_conflict since: 0.3.0-rc.5 - covers: [merge.path_merge, merge.conflict_reporting] + covers: [merge/0.1.path_merge, merge/0.1.conflict_reporting] operation: merge_records input: paths: @@ -1599,7 +1599,7 @@ groups: - name: "a column-0 comment inside a block sequence belongs to the entry" id: merge.column0_comment_inside_sequence since: 0.3.0-rc.5 - covers: [merge.merge_format_fidelity] + covers: [merge/0.1.merge_format_fidelity] operation: merge_records input: path: items/a.md @@ -1650,7 +1650,7 @@ groups: - name: "append-append onto a base without a final line break adds no empty line" id: merge.append_unterminated_base since: 0.3.0-rc.5 - covers: [merge.append_append_union] + covers: [merge/0.1.append_append_union] operation: merge_records input: path: items/a.md @@ -1687,7 +1687,7 @@ groups: - name: "append-append in a CRLF body inserts a CRLF separator" id: merge.append_separator_crlf since: 0.3.0-rc.5 - covers: [merge.append_append_union] + covers: [merge/0.1.append_append_union] operation: merge_records input: path: items/a.md @@ -1700,7 +1700,7 @@ groups: - name: "the body alignment is the pinned Myers alignment" id: merge.body_alignment_pinned since: 0.3.0-rc.5 - covers: [merge.body_three_way] + covers: [merge/0.1.body_three_way] operation: merge_records input: path: items/a.md @@ -1738,7 +1738,7 @@ groups: - name: "plain scalars use the YAML 1.2 core schema: 0777 is decimal" id: merge.yaml_core_schema_decimal since: 0.3.0-rc.5 - covers: [merge.max_min_semantics] + covers: [merge/0.1.max_min_semantics] operation: merge_records input: path: items/a.md @@ -1771,7 +1771,7 @@ groups: - name: "plain scalars use the YAML 1.2 core schema: yes is a string" id: merge.yaml_core_schema_yes since: 0.3.0-rc.5 - covers: [merge.field_three_way, merge.conflict_reporting] + covers: [merge/0.1.field_three_way, merge/0.1.conflict_reporting] operation: merge_records input: path: items/a.md @@ -1809,7 +1809,7 @@ groups: - name: frontmatter that is not a mapping merges as one unit id: merge.non_mapping_frontmatter since: 0.3.0-rc.5 - covers: [merge.conflict_reporting] + covers: [merge/0.1.conflict_reporting] operation: merge_records input: path: items/a.md @@ -1839,7 +1839,7 @@ groups: - name: declared and default strategies for one record id: merge.strategies_item since: 0.3.0-rc.5 - covers: [merge.default_strategies, merge.declared_strategies, collection_semantics.merge_declarations] + covers: [merge/0.1.default_strategies, merge/0.1.declared_strategies, collection_semantics.merge_declarations] operation: merge_strategies input: path: items/a.md @@ -1928,7 +1928,7 @@ groups: - name: "a type_conflict between declarations merges the field as a conflict" id: merge.declaration_type_conflict_in_merge since: 0.3.0-rc.5 - covers: [collection_semantics.merge_declarations, merge.conflict_reporting] + covers: [collection_semantics.merge_declarations, merge/0.1.conflict_reporting] operation: merge_records input: path: shared/x.md diff --git a/tests/v0.3/watch/move-detection.yaml b/tests/v0.3/watch/move-detection.yaml index fb137c4..44275e6 100644 --- a/tests/v0.3/watch/move-detection.yaml +++ b/tests/v0.3/watch/move-detection.yaml @@ -1,4 +1,4 @@ -# Move detection (Chapter 12A). Each test lists what disappeared and what +# Move detection (concurrent edits profile 0.1). Each test lists what disappeared and what # appeared within one observation window. file_id stands for a platform file # identity such as a device and inode; omit it when the platform cannot supply # one. Cases follow the identity measurements in the mdbase-next prototype @@ -6,16 +6,16 @@ # same bytes, delete and create of an edited file, and unrelated files. name: "move detection" spec_version: "0.3.0" -fixture_set: watch +fixture_set: move_detection category: watch -spec_ref: v0.3/12A +spec_ref: concurrent-edits/0.1 groups: - name: pairing disappearances and appearances within one observation window tests: - name: a rename with unchanged bytes is a move id: watch.moves_rename_same_content since: 0.3.0-rc.5 - covers: [watch.move_detection] + covers: [move_detection/0.1.move_pairing] operation: detect_moves input: disappeared: @@ -59,7 +59,7 @@ groups: - name: a rename followed by an edit keeps the file identity id: watch.moves_rename_then_edit since: 0.3.0-rc.5 - covers: [watch.move_detection] + covers: [move_detection/0.1.move_pairing] operation: detect_moves input: disappeared: @@ -102,7 +102,7 @@ groups: - name: "delete and create with identical bytes is a move (git checkout)" id: watch.moves_delete_create_same_bytes since: 0.3.0-rc.5 - covers: [watch.move_detection] + covers: [move_detection/0.1.move_pairing] operation: detect_moves input: disappeared: @@ -146,7 +146,7 @@ groups: - name: delete and create of an edited file with the same name in another folder is a move id: watch.moves_same_name_edited since: 0.3.0-rc.5 - covers: [watch.move_detection] + covers: [move_detection/0.1.move_pairing] operation: detect_moves input: disappeared: @@ -190,7 +190,7 @@ groups: - name: the same name with similarity below 0.8 and no file identity is not a move id: watch.moves_same_name_too_different since: 0.3.0-rc.5 - covers: [watch.move_detection] + covers: [move_detection/0.1.move_pairing] operation: detect_moves input: disappeared: @@ -229,7 +229,7 @@ groups: - name: an unrelated delete and create are never paired id: watch.moves_unrelated_files since: 0.3.0-rc.5 - covers: [watch.move_detection] + covers: [move_detection/0.1.move_pairing] operation: detect_moves input: disappeared: @@ -264,7 +264,7 @@ groups: - name: equal id_field values pair even when the content changed completely id: watch.moves_identity_hint_pairs since: 0.3.0-rc.5 - covers: [watch.move_detection] + covers: [move_detection/0.1.move_pairing] operation: detect_moves input: id_field: id @@ -293,7 +293,7 @@ groups: - name: "different id_field values never pair, even with the same name and file identity" id: watch.moves_identity_hint_vetoes since: 0.3.0-rc.5 - covers: [watch.move_detection] + covers: [move_detection/0.1.move_pairing] operation: detect_moves input: id_field: id @@ -322,7 +322,7 @@ groups: - name: "when two appearances match equally, the smaller path pairs" id: watch.moves_copies_pair_deterministically since: 0.3.0-rc.5 - covers: [watch.move_detection] + covers: [move_detection/0.1.move_pairing] operation: detect_moves input: disappeared: @@ -378,7 +378,7 @@ groups: - name: identical bytes outrank a same-name match id: watch.moves_identical_bytes_beat_name since: 0.3.0-rc.5 - covers: [watch.move_detection] + covers: [move_detection/0.1.move_pairing] operation: detect_moves input: disappeared: From 91c2404031a6be3e189c52b4f462d32adcaf8ad9 Mon Sep 17 00:00:00 2001 From: callumalpass Date: Thu, 12 Nov 2026 07:04:12 +1100 Subject: [PATCH 4/7] Move gateway access and approval rules to the interop profile Chapter 05A keeps what a collection tool needs: a lookup returns every implementation, none is chosen silently, and implementing a contract grants no access. What a gateway exposes and what an approval pins is section 10 of interop/0.1.md. --- 05a-data-contracts.md | 42 ++++++++++----------------------------- interop/0.1.md | 46 ++++++++++++++++++++++++++++++++++++------- 2 files changed, 49 insertions(+), 39 deletions(-) diff --git a/05a-data-contracts.md b/05a-data-contracts.md index 6853aa1..33e2391 100644 --- a/05a-data-contracts.md +++ b/05a-data-contracts.md @@ -311,38 +311,16 @@ arbitrarily selected provider. Multiple applications may consume the same type implementation. Implementing a contract does not create an owner, lease, or exclusive provider relationship. -When several types implement one compatible contract requirement: - -- read and list experiences SHOULD initially offer their explicit union -- user-facing approval MUST show every included type -- an existing approval MUST pin the exact type names, contract digest, and - implementation digests -- a later implementation MUST NOT silently join that approval -- creation MUST use one explicitly selected implementing type - -A product may let a user select a subset instead of the union. It MUST NOT hide -the selection or silently choose the first filesystem entry. - -These rules separate interoperability from authorization. A type's -`implements` claim says what it can mean; it does not say which application may -read or mutate it. - -## Contract Access And Whole Records - -The portable contract view contains only mapped contract fields. A gateway that -grants access "through a contract" MUST expose only that view plus the minimum -record identity needed by its protocol. If a record has several approved views -and the operation does not identify one unambiguously, the gateway MUST require -an explicit contract ID, exact version, and implementing type rather than merge -or guess. - -Access to unmapped frontmatter, the Markdown body, or arbitrary records is -whole-record or whole-collection access and MUST be requested and presented -explicitly. Merely implementing a contract never grants either form of access. - -Core collection APIs remain authorization-neutral. This distinction is -normative for gateways and application protocols that use data contracts as an -authorization boundary. +A tool MUST NOT silently choose one of several implementations, for example +the first filesystem entry. A create through a contract names one +implementing type. + +A type's `implements` claim says what the type can mean; it does not say which +application may read or mutate it. Collection APIs are authorization-neutral, +and implementing a contract grants no access. The rules for a gateway that +grants access through a contract, including what an approval shows and pins, +are in the +[event/action interoperability profile](./interop/0.1.md). ## Diagnostics diff --git a/interop/0.1.md b/interop/0.1.md index 19630aa..1067497 100644 --- a/interop/0.1.md +++ b/interop/0.1.md @@ -14,6 +14,8 @@ The profile defines: - event-source and action-provider declarations; - discovery, version resolution, validation, authorization boundaries, and duplicate semantics; +- what a gateway exposes and pins when it grants record access through a + contract; - role-specific conformance; - transport capabilities and the baseline bridge behavior. @@ -345,7 +347,37 @@ when necessary. Unauthorized publish, subscribe, provider registration, and invocation fail closed. -## 10. Duplicate, Idempotency, and Indeterminate Outcomes +## 10. Record Contract Access and Approval + +A **gateway** is a bridge, or another service, that grants an application +access to records through a record contract. Collection APIs are +authorization-neutral; these rules are normative for gateways and application +protocols that use data contracts as an authorization boundary. + +When several types implement one compatible contract requirement: + +- read and list experiences SHOULD initially offer their explicit union +- user-facing approval MUST show every included type +- an existing approval MUST pin the exact type names, contract digest, and + implementation digests +- a later implementation MUST NOT silently join that approval +- creation MUST use one explicitly selected implementing type + +A product may let a user select a subset instead of the union. It MUST NOT hide +the selection or silently choose the first filesystem entry. + +The portable contract view contains only mapped contract fields. A gateway that +grants access "through a contract" MUST expose only that view plus the minimum +record identity needed by its protocol. If a record has several approved views +and the operation does not identify one unambiguously, the gateway MUST require +an explicit contract ID, exact version, and implementing type rather than merge +or guess. + +Access to unmapped frontmatter, the Markdown body, or arbitrary records is +whole-record or whole-collection access and MUST be requested and presented +explicitly. Merely implementing a contract never grants either form of access. + +## 11. Duplicate, Idempotency, and Indeterminate Outcomes The profile does not claim universal exactly-once behavior. @@ -363,7 +395,7 @@ The profile does not claim universal exactly-once behavior. Retry policy belongs to the caller, binding, or durable runtime. -## 11. Cancellation and Deadlines +## 12. Cancellation and Deadlines A caller may request cancellation by `request_id`. Cancellation is cooperative. The bridge verifies caller ownership and selected-provider @@ -384,7 +416,7 @@ a cooperative provider when the deadline expires during an attempt. A binding that cannot enforce a required deadline fails `unsupported_transport_capability`. -## 12. Transport Capabilities +## 13. Transport Capabilities A binding describes: @@ -407,7 +439,7 @@ the same process. It still prevents accidental cross-application registration through opaque client handles and validates host-attributed identities at each boundary. -## 13. Baseline Bridge API +## 14. Baseline Bridge API The behavior-based API is: @@ -428,7 +460,7 @@ Language SDKs may use idiomatic names. A client handle scopes every subscription and provider registration to one implementation and removes them on disposal. -## 14. Standard Errors +## 15. Standard Errors Portable error codes are defined by `schemas/interop/v0.1/profile.schema.json#/$defs/portableError`. @@ -437,7 +469,7 @@ Transport-private diagnostics may be nested in safe `details`, but they do not replace the portable category. Error messages MUST NOT expose credentials, private authorization policy, or provider-internal stack traces. -## 15. Relationship to Runtime and Hosted Bindings +## 16. Relationship to Runtime and Hosted Bindings The [mdbase durable runtime profile](../runtime/0.2.md) consumes these contract artifacts and envelopes. It @@ -450,7 +482,7 @@ envelopes. Its control-plane wake-up signals may remain opaque: private event data can stay at the collection authority and be retrieved only by an authorized application. -## 16. Conformance +## 17. Conformance Required scenarios are enumerated in `tests/v0.3/event-action-interop/event-action-interop.yaml`. Claims validate From 71769178c8573857f6f96a05d68d887ed0580cb7 Mon Sep 17 00:00:00 2001 From: callumalpass Date: Thu, 12 Nov 2026 07:05:54 +1100 Subject: [PATCH 5/7] Define the regex profile and merge line alignment without reference to an implementation The regex-lite crate and diff-match-patch's bisect become informative notes. The line alignment of the body merge is written out as a procedure, checked against the executable model. --- 10-cel-profile.md | 29 ++++++----- concurrent-edits/0.1.md | 104 ++++++++++++++++++++++++++++++---------- 2 files changed, 97 insertions(+), 36 deletions(-) diff --git a/10-cel-profile.md b/10-cel-profile.md index 2a79e2d..a159574 100644 --- a/10-cel-profile.md +++ b/10-cel-profile.md @@ -318,9 +318,11 @@ pattern the same way on every platform, which matters because a tool that replays or re-verifies a write must reach the same result as the tool that made it. -The profile is RE2 syntax with ASCII-only character classes, the semantics of -the Rust `regex-lite` crate. CEL specifies RE2 syntax for `matches()`; the -profile keeps that syntax and removes Unicode classes from it. +The profile is RE2 syntax with ASCII-only character classes. CEL specifies +RE2 syntax for `matches()`; the profile keeps that syntax and removes Unicode +classes from it. This section defines the profile completely: the syntax +list, the limits, and the semantics below are normative, and no +implementation's behavior is. **Syntax.** A pattern may use: @@ -341,14 +343,14 @@ profile keeps that syntax and removes Unicode classes from it. A pattern that uses anything else is invalid. In particular, Unicode classes such as `\p{L}` and `\pN`, backreferences, and look-around are invalid, and -so are constructs that `regex-lite` accepts beyond this list: `\b{start}` and +so are these constructs, which some RE2-family engines accept: `\b{start}` and the other `\b{...}` boundaries, `(?...)` (use `(?P...)`), the `U` and `R` flags, escapes of letters or digits other than those listed (such as `\a` and `\u0041`), nested bracket classes, and class set operations (`&&`, `--`, `~~`). A `{` that does not begin a valid repetition, and a `]` or `}` outside a class, are invalid too; escape them. `\<` and `\>` are -literal `<` and `>`, as the escape rule above says, although `regex-lite` -itself reads them as word boundaries. +literal `<` and `>`, as the escape rule above says, although some engines +read them as word boundaries. **Limits.** Every tool accepts and rejects the same patterns: @@ -363,8 +365,8 @@ itself reads them as word boundaries. `+`, and `?`; and the operand times `max(n, m, 1) + 1` for `{n}`, `{n,}`, and `{n,m}`. -`regex-lite`'s own size limit depends on the platform's pointer width, so it -cannot be the portable limit. +An engine's own size limit usually depends on the platform, for example on +its pointer width, so it cannot be the portable limit. **Semantics.** Matching runs over Unicode scalar values, and a match is unanchored unless the pattern anchors it, as in RE2 and JSON Schema. Among @@ -391,10 +393,13 @@ during evaluation that turns out invalid raises an evaluation error. An invalid JSON Schema `pattern` or `match.where` pattern makes its type file invalid. -**Provisional (rc.5).** The profile follows `regex-lite` because it keeps the -WebAssembly runtime about 740 KB smaller than a Unicode-table engine and can -run identically everywhere. Patterns that depend on Unicode classes behave -differently from rc.4 engines that used a Unicode-aware engine. +*Note (informative).* The profile was drawn from the Rust `regex-lite` +crate, which needs no Unicode tables: an engine built on it is several +hundred kilobytes smaller as WebAssembly and runs identically everywhere. An +implementation that uses `regex-lite` still applies the syntax restrictions +and limits above itself, because the crate accepts more. Patterns that +depend on Unicode classes behave differently from rc.4 engines that used a +Unicode-aware engine. ## File And Link Helpers diff --git a/concurrent-edits/0.1.md b/concurrent-edits/0.1.md index 49771ca..865152d 100644 --- a/concurrent-edits/0.1.md +++ b/concurrent-edits/0.1.md @@ -244,30 +244,86 @@ standing for the three bodies: differently, the body is in conflict. When the body is in conflict, the merged body is `F` as a whole and the -conflict carries the three bodies. The alignment of `B` with each side is a -longest common subsequence of lines, chosen as follows, so that every -implementation splits chunks the same way: - -1. Lines common to the start of both sequences are aligned, then lines - common to their end. -2. The remaining middle parts are split at the **middle snake** of Myers' - linear-space algorithm (E. Myers, "An O(ND) Difference Algorithm and Its - Variations", 1986, section 4b), in the formulation of diff-match-patch's - `bisect`: for each edit distance `d` the forward search runs before the - reverse search; on each diagonal a search continues from the neighbouring - diagonal with the larger furthest-reaching value, and from the lower - diagonal (a deletion) when they are equal; the split point is the forward - search's furthest-reaching point on the diagonal where the searches first - overlap. -3. Each part is aligned recursively with these rules. - -The executable model (`scripts/concurrent_edits_model.py`, `_lcs_pairs`) -is the reference for this procedure. - -**Provisional (rc.5).** Earlier drafts left the choice among several longest -common subsequences open. The procedure above is the one the reference model -and the first engine implement. A later release may name a simpler canonical -alignment if one proves as fast. +conflict carries the three bodies. + +### Line alignment + +The alignment of `B` with a side is a list of pairs of equal lines, one from +each sequence, in increasing order in both. Several such lists can have the +greatest possible length, and they split a body into different chunks, so +every implementation computes the alignment of two sequences `X` and `Y` +with exactly this procedure: + +1. While both sequences are non-empty and their first lines are equal, those + lines are a pair, and both are removed. +2. While both sequences are non-empty and their last lines are equal, those + lines are a pair, and both are removed. +3. If either sequence is now empty, the alignment is complete. +4. Otherwise the **split point** below gives a number of lines `x` of `X` + and `y` of `Y`. The first `x` lines of `X` are aligned with the first `y` + lines of `Y`, and the remaining lines of `X` with the remaining lines of + `Y`, each with this procedure. If there is no split point, the remaining + lines have no pairs. + +The split point of `X`, with `n` lines, and `Y`, with `m` lines, is found by +searching from both ends at once. Lines are numbered from 0. `F` and `R` +each map a diagonal, which is an integer, to a number of lines. An entry +that has not been assigned is unset, and an unset entry reads as -1 in a +comparison. + +```text +limit = ceil((n + m) / 2) +delta = n - m +F[1] = 0; R[1] = 0 +f_low = 0; f_high = 0; r_low = 0; r_high = 0 + +for d = 0, 1, ..., limit - 1: + + for k = -d + f_low, -d + f_low + 2, ..., while k <= d - f_high: + if k == -d, or (k != d and F[k - 1] < F[k + 1]): + x = F[k + 1] + else: + x = F[k - 1] + 1 + y = x - k + while x < n and y < m and X[x] == Y[y]: + x = x + 1; y = y + 1 + F[k] = x + if x > n: + f_high = f_high + 2 + else if y > m: + f_low = f_low + 2 + else if delta is odd and R[delta - k] is set + and x >= n - R[delta - k]: + return (x, y) + + for k = -d + r_low, -d + r_low + 2, ..., while k <= d - r_high: + if k == -d, or (k != d and R[k - 1] < R[k + 1]): + x = R[k + 1] + else: + x = R[k - 1] + 1 + y = x - k + while x < n and y < m and X[n - x - 1] == Y[m - y - 1]: + x = x + 1; y = y + 1 + R[k] = x + if x > n: + r_high = r_high + 2 + else if y > m: + r_low = r_low + 2 + else if delta is even and F[delta - k] is set + and F[delta - k] >= n - x: + return (F[delta - k], F[delta - k] - (delta - k)) + +no split point +``` + +*Note (informative).* This is the middle snake of Myers' linear-space +algorithm (E. Myers, "An O(ND) Difference Algorithm and Its Variations", +1986, section 4b), in the form that the diff-match-patch library uses in +`bisect`. `scripts/concurrent_edits_model.py` implements it in +`_lcs_pairs`, and the merge fixtures run against that model. + +**Provisional (0.1).** A later version of this profile may name a simpler +canonical alignment if one proves as fast. **Provisional (rc.5).** Append-append applies only to appends at the end of the body. Two insertions at the same place inside the body remain a From ffbaf685dd4b3e741c3c7aae3f692f5165d7a851 Mon Sep 17 00:00:00 2001 From: callumalpass Date: Thu, 12 Nov 2026 07:05:55 +1100 Subject: [PATCH 6/7] Move path equivalence from Core Read to Collection Semantics Core Read is now a per-record profile: it computes no path keys and reports no path_collision. --- 02-collection-layout.md | 8 +++++--- 14-conformance.md | 8 ++++---- tests/v0.3/core/list-ops-and-fidelity.yaml | 2 +- tests/v0.3/core/paths.yaml | 6 +++--- tests/v0.3/manifest.yaml | 2 +- 5 files changed, 14 insertions(+), 12 deletions(-) diff --git a/02-collection-layout.md b/02-collection-layout.md index f2cabc6..34d5c82 100644 --- a/02-collection-layout.md +++ b/02-collection-layout.md @@ -225,9 +225,11 @@ collides, whatever file system it runs on: `tasks/todo.md` to `tasks/Todo.md`, changes only the spelling of the path and is not a collision. - When record discovery finds several files with one path key, which can - happen on a case-sensitive file system, each file is still a record. Core - Read reports a `path_collision` warning on every record of the group, with - `details.paths` listing the group in code-point order. + happen on a case-sensitive file system, each file is still a record. A + tool that claims Collection Semantics reports a `path_collision` warning + on every record of the group, with `details.paths` listing the group in + code-point order. Core Read alone reads each file as written and computes + no path keys. **Provisional (rc.5).** A tool that keeps records in a log or replicates them across file systems cannot hold two records with one path key. It diff --git a/14-conformance.md b/14-conformance.md index d682878..a841e66 100644 --- a/14-conformance.md +++ b/14-conformance.md @@ -8,8 +8,8 @@ queries, writes, runtime preflight, workflow execution, and watching. | Profile | Purpose | | --- | --- | -| Core Read | discover collections, parse records, load types, match records, and validate JSON Schema | -| Collection Semantics | apply defaults, uniqueness, path policy, multi-type composition, and collection diagnostics | +| Core Read | discover collections, parse records, load types, match records, and validate JSON Schema, one record at a time | +| Collection Semantics | apply defaults, uniqueness, path equivalence, path policy, multi-type composition, and collection diagnostics | | Data Contracts | load contracts, resolve type implementations, compute digests, and project contract views | | CEL | compile and evaluate the shared mdbase CEL language and host contract | | CEL Match | evaluate `match.expr` against raw candidate records | @@ -203,8 +203,6 @@ Core Read implementations MUST: `unsupported_profile` - report diagnostics in the canonical machine-readable shape - read, index, and report invalid records rather than skip them -- compute path keys and report `path_collision` for discovered records with - equivalent paths ## Data Contracts Requirements @@ -230,6 +228,8 @@ Collection Semantics implementations MUST: comparison set, comparing raw values without coercion - report cross-record issues without blocking writes, except for `enforce: write` uniqueness rules +- compute path keys and report `path_collision` for discovered records with + equivalent paths - load `collection.merge` declarations and derive default merge strategies - validate portable `collection.path` policies for write-capable tools - decode path-stored fields from record paths and use the decoded values diff --git a/tests/v0.3/core/list-ops-and-fidelity.yaml b/tests/v0.3/core/list-ops-and-fidelity.yaml index d8daadc..cf5e499 100644 --- a/tests/v0.3/core/list-ops-and-fidelity.yaml +++ b/tests/v0.3/core/list-ops-and-fidelity.yaml @@ -361,7 +361,7 @@ groups: - name: "both files are records and both report path_collision" id: paths.discovered_collision since: 0.3.0-rc.5 - covers: [core_read.path_equivalence] + covers: [collection_semantics.path_equivalence] operation: read input: path: "notes/plan.md" diff --git a/tests/v0.3/core/paths.yaml b/tests/v0.3/core/paths.yaml index 84b6c8f..5fc59cc 100644 --- a/tests/v0.3/core/paths.yaml +++ b/tests/v0.3/core/paths.yaml @@ -12,7 +12,7 @@ groups: - name: paths that differ in case or Unicode normalization share a path key id: paths.case_and_nfc since: 0.3.0-rc.5 - covers: [core_read.path_equivalence] + covers: [collection_semantics.path_equivalence] operation: path_equivalence input: paths: ["Notes/Café.md", "notes/café.md", "NOTES/CAFÉ.md", notes/other.md] @@ -22,7 +22,7 @@ groups: - name: "full case folding makes Straße and STRASSE equivalent" id: paths.full_case_folding since: 0.3.0-rc.5 - covers: [core_read.path_equivalence] + covers: [collection_semantics.path_equivalence] operation: path_equivalence input: paths: ["Straße.md", STRASSE.md, strasse-2.md] @@ -32,7 +32,7 @@ groups: - name: different names are not equivalent id: paths.distinct_paths since: 0.3.0-rc.5 - covers: [core_read.path_equivalence] + covers: [collection_semantics.path_equivalence] operation: path_equivalence input: paths: [tasks/a.md, tasks/b.md, tasks/a.base] diff --git a/tests/v0.3/manifest.yaml b/tests/v0.3/manifest.yaml index 9f3778b..34813b7 100644 --- a/tests/v0.3/manifest.yaml +++ b/tests/v0.3/manifest.yaml @@ -32,7 +32,6 @@ claim_profiles: - raw_schema_validation - machine_readable_diagnostics - invalid_records_reported # since 0.3.0-rc.5 - - path_equivalence # since 0.3.0-rc.5 - regex_profile # since 0.3.0-rc.5 - frontmatter_delimiters # since 0.3.0-rc.6 - id: collection_semantics @@ -49,6 +48,7 @@ claim_profiles: - cross_record_reporting # since 0.3.0-rc.5 - merge_declarations # since 0.3.0-rc.5 - path_stored_fields # since 0.3.0-rc.6 + - path_equivalence # since 0.3.0-rc.5; core_read.path_equivalence before 0.3.0-rc.6 - id: data_contracts status: coverage_complete requires: [core_read] From 26ee1ea1742f3a07616628a81175aa5aeda13d60 Mon Sep 17 00:00:00 2001 From: callumalpass Date: Thu, 12 Nov 2026 07:07:44 +1100 Subject: [PATCH 7/7] Close the rc series: stable criteria, exact spec_version, provisional checklist rc.6 is the last release candidate that adds or changes behavior. 0.3.0 is stable when no Provisional paragraph remains in chapters 00-14 and the release contract's implementations hold verified claims. spec_version is exactly 0.3.0. Chapter 13 lists the rc.6 rules and profile ID changes. --- 00-overview.md | 39 +++++++++++++++++++++++------- 04-configuration.md | 5 +++- 13-migrations-and-compatibility.md | 34 +++++++++++++++++++++++++- CHANGELOG.md | 38 +++++++++++++++++++++++++++++ README.md | 2 +- concurrent-edits/0.1.md | 14 +++++------ release/v0.3.0.md | 22 +++++++++++++++++ 7 files changed, 135 insertions(+), 19 deletions(-) diff --git a/00-overview.md b/00-overview.md index 4e1ddd1..7cc0337 100644 --- a/00-overview.md +++ b/00-overview.md @@ -344,11 +344,25 @@ directly comparable without making one product's internal API normative. ## Versioning This specification uses semantic versioning. The current version is **0.3.0**, -in its fifth release candidate (`0.3.0-rc.5`); the sixth (`0.3.0-rc.6`) is -in draft. 0.3.0 is declared stable once -implementations pass the conformance suite. +in its sixth release candidate (`0.3.0-rc.6`), which is in draft. Collections declare their specification version with `spec_version` in -`mdbase.yaml`. Tools declare the profiles and versions they implement. +`mdbase.yaml`, which is `0.3.0` for every release candidate. Tools declare +the profiles and versions they implement. + +rc.6 is the last release candidate that adds or changes behavior. A later +release candidate only confirms or removes a provisional choice, corrects an +error, or adds tests. 0.3.0 is declared stable when both of these hold: + +1. No paragraph marked **Provisional** remains in chapters 00 to 14. Each is + confirmed, by removing its marker, or removed together with the behavior + it describes. +2. The implementations named in the + [release contract](./release/v0.3.0.md) hold verified conformance claims + (Chapter 14) against that release candidate for the profiles of this + specification that the contract lists for them. + +Companion profiles are not part of that gate. They keep their own versions +and can change after 0.3.0 is stable. Companion profiles, artifacts, and schemas carry their own versions: @@ -375,11 +389,18 @@ The keywords `MUST`, `MUST NOT`, `SHOULD`, `SHOULD NOT`, and `MAY` are to be interpreted as described in RFC 2119. Draft notes use ordinary prose and are non-normative. A paragraph that begins -with **Provisional (rc.5).** or **Provisional (rc.6).** records a choice made where the design input -left a detail open. It is normative in the release candidate and may change -before 0.3.0 is declared stable. The -[0.3.0-rc.5 release notes](./docs/releases/0.3.0-rc.5.md) list every such -choice. +with **Provisional (rc.5).** or **Provisional (rc.6).** records a choice made +where the design input left a detail open. It is normative in the release +candidate, and it is confirmed or removed before 0.3.0 is declared stable +(see Versioning). The [release contract](./release/v0.3.0.md) lists every +such paragraph. A companion profile marks such a choice with its own version, +as in **Provisional (0.1).**, and may change it in a later version of the +profile. + +A paragraph that begins with *Note (informative).* is not normative. Where +the specification names a library or an implementation, it does so only in +such a note: the text of the specification, not the behavior of any +implementation, defines conformance. ## License diff --git a/04-configuration.md b/04-configuration.md index e7fd08c..777ba2c 100644 --- a/04-configuration.md +++ b/04-configuration.md @@ -32,7 +32,10 @@ settings: is the compatibility boundary. A v0.3 tool MUST reject v0.2 and v0.4 collections unless an explicit compatibility adapter is enabled. -Pre-1.0 draft versions MAY be accepted by explicit compatibility setting. +The value is exactly `"0.3.0"`, throughout the release-candidate series and +after it. A release candidate is a version of this specification, which a +tool names in its conformance claim (Chapter 14); it is never a version of a +collection, and a value such as `0.3.0-rc.5` is invalid. ## Settings diff --git a/13-migrations-and-compatibility.md b/13-migrations-and-compatibility.md index 6b7f02b..1617010 100644 --- a/13-migrations-and-compatibility.md +++ b/13-migrations-and-compatibility.md @@ -112,7 +112,39 @@ write sets the title, which removes the key. An engine moving from rc.5 to rc.6 decodes path-stored fields on read, never writes them to frontmatter, derives paths from them after lifecycle, moves a record when an update changes one, and reports `unknown_path_key`, -`path_value_inconsistent`, and `path_field_in_frontmatter`. +`path_value_inconsistent`, `path_field_in_frontmatter`, and +`path_value_normalized`. + +rc.6 also writes down rules that rc.5 left to the implementation. An engine +checks its behavior against each: + +| Area | rc.6 rule | +| --- | --- | +| `slugify` (Chapter 09) | NFD without combining marks, ASCII lowercase, other runs to `-`, trimmed; no transliteration | +| link targets (Chapter 08) | compared with paths and filenames by path key; IDs exactly | +| Markdown link destinations (Chapter 08) | in angle brackets as written, or up to the first space or `)` with only `%20` decoded | +| inline tags and code in the body (Chapter 08) | one tag grammar; one rule for fenced blocks, inline code, and backslashes, shared by links and tags | +| frontmatter delimiters (Chapter 03) | only a line that is exactly `---` closes frontmatter | +| create (Chapter 12) | membership is frozen at the membership path before lifecycle runs | +| `order_by` (Chapter 11) | one order of value kinds; the final tie-break is ascending `file.path` | + +### Profiles in rc.6 + +rc.6 moves optional behavior out of the chapters into companion profiles +with their own versions, and renames the profile IDs to match. No behavior +changes except where the table says so. A conformance claim written for rc.5 +changes its profile IDs as follows: + +| rc.5 | rc.6 | Change | +| --- | --- | --- | +| `merge` (Chapter 12A) | `merge/0.1` ([concurrent edits profile](./concurrent-edits/0.1.md)) | rebasing `body_edits` onto a changed body is required here, no longer by `core_write` | +| `watch` requirement `move_detection` (Chapter 12A) | `move_detection/0.1` ([concurrent edits profile](./concurrent-edits/0.1.md)) | `watch` alone reports an externally moved file as a delete and a create | +| `type_packs` (Chapter 05A) | `type_packs/0.1` ([type packs profile](./type-packs/0.1.md)) | none | +| `core_read` requirement `path_equivalence` | `collection_semantics` requirement `path_equivalence` | Core Read computes no path keys and reports no `path_collision` | + +Record identity is now in Chapter 03 and writer format fidelity in Chapter +12. The gateway access and approval rules of Chapter 05A are now section 10 +of the [event/action interoperability profile](./interop/0.1.md). ## From v0.2 To v0.3 diff --git a/CHANGELOG.md b/CHANGELOG.md index ee87d00..248dc74 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -115,6 +115,44 @@ Eight places where two conforming tools could disagree. See order whatever the directions of the terms, and applies to tools that page with their own continuation tokens. +### Shape of the specification + +#### Changed + +- Chapter 12A is divided. Move detection and the three-way record merge are + the [concurrent edits profile 0.1](./concurrent-edits/0.1.md), with profile + IDs `move_detection/0.1` (requires `watch`) and `merge/0.1` (requires + `core_write`). Record identity is in Chapter 03 and writer format fidelity + in Chapter 12. The moved text is unchanged. +- Type packs move from Chapter 05A to the + [type packs profile 0.1](./type-packs/0.1.md). The profile ID is + `type_packs/0.1`. +- The rules for a gateway that grants record access through a contract, and + for what an approval shows and pins, move from Chapter 05A to section 10 + of the interoperability profile. Its later sections are renumbered. +- Core Write no longer requires the body merge. An implementation that does + not claim `merge/0.1` applies `body_edits` only when the body matches + `body_base` and otherwise reports `body_base_unavailable`. +- Watch no longer requires move detection. Only an implementation that + claims `move_detection/0.1` reports an externally moved file as + `record_renamed`; every other reports a delete and a create. +- Path equivalence moves from Core Read to Collection Semantics + (`collection_semantics.path_equivalence`). Core Read computes no path keys + and reports no `path_collision`. +- The regex profile (Chapter 10) and the line alignment of the body merge + are defined by the text alone. The `regex-lite` crate and + diff-match-patch's `bisect` are named only in informative notes, and the + alignment is written out as a procedure. +- `spec_version` is exactly `"0.3.0"`. The sentence in Chapter 04 that + allowed draft versions by a compatibility setting is removed. +- Chapter 00 states how the release-candidate series closes: rc.6 is the + last one that adds or changes behavior, and 0.3.0 is stable when no + Provisional paragraph remains in chapters 00 to 14 and the implementations + of the release contract hold verified claims. The release contract lists + the thirteen remaining Provisional paragraphs. +- The fixture set `watch` is renamed `move_detection`. Fixture files keep + their paths. + ## 0.3.0-rc.5 (draft, untagged) The fifth release candidate specifies the data semantics that let several diff --git a/README.md b/README.md index 3f5de19..45f6fbb 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ update files safely. The durable data stays in ordinary Markdown files. Collections remain readable in a text editor, reviewable in Git, and usable across conforming tools. -The current specification is **v0.3.0** (release candidate 5). +The current specification is **v0.3.0** (release candidate 6, in draft). [Read the overview](./00-overview.md) · [Visit mdbase.dev](https://mdbase.dev) · diff --git a/concurrent-edits/0.1.md b/concurrent-edits/0.1.md index 865152d..1257c30 100644 --- a/concurrent-edits/0.1.md +++ b/concurrent-edits/0.1.md @@ -94,9 +94,9 @@ A tool reports a detected move as `record_renamed` (Chapter 14), followed by `record_modified` when the bytes also changed. Internal identity, history, and merge bases follow the record to its new path. -**Provisional (rc.5).** The 5-second minimum window and the 0.5 and 0.8 -thresholds come from the feasibility prototype's measurements and may be -tuned before release. +**Provisional (0.1).** The 5-second minimum window and the 0.5 and 0.8 +thresholds come from measurements of a feasibility prototype and may be +tuned in a later version of this profile. ## Three-Way Record Merge @@ -201,10 +201,10 @@ or `min` result written in a different form, is re-emitted. The merged frontmatter source starts from the first version's source. Comment and blank lines between entries therefore come from the first version. -**Provisional (rc.5).** Comment or blank lines between entries that only +**Provisional (0.1).** Comment or blank lines between entries that only the second version changed are not carried into the merged version, unless the -first version has the same bytes as the base. A later release candidate may merge them as -text. +first version has the same bytes as the base. A later version of this +profile may merge them as text. When the frontmatter of any of the three versions is not a mapping, the whole frontmatter is one unit: the result follows rules 1 to 3 above on its source @@ -325,7 +325,7 @@ algorithm (E. Myers, "An O(ND) Difference Algorithm and Its Variations", **Provisional (0.1).** A later version of this profile may name a simpler canonical alignment if one proves as fast. -**Provisional (rc.5).** Append-append applies only to appends at the end +**Provisional (0.1).** Append-append applies only to appends at the end of the body. Two insertions at the same place inside the body remain a conflict. diff --git a/release/v0.3.0.md b/release/v0.3.0.md index b6cf9a5..eb94cd1 100644 --- a/release/v0.3.0.md +++ b/release/v0.3.0.md @@ -123,6 +123,28 @@ passes preflight. - [ ] No unresolved normative questions in chapters 00-14, Chapter 05A, and the companion documents after the spec review cleanup (standard CEL, date strings, `unset`, batch, validation levels, links, and profile split). +- [ ] `0.3.0-rc.6` is tagged. It is the last release candidate that adds or + changes behavior. +- [ ] No **Provisional** paragraph remains in chapters 00-14. Each one below + is confirmed, by removing its marker, or removed with its behavior: + - [ ] 02: path safety rule set + - [ ] 02: resolving a discovered `path_collision` group by renaming + - [ ] 02: full case folding in path keys + - [ ] 02: Unicode 17.0.0 data for NFC and case folding + - [ ] 03: YAML scalar resolution table + - [ ] 04: data contract view failures classed as single-record + - [ ] 06: `pattern` uses the mdbase regex profile, not ECMA-262 + - [ ] 07: `enforce: write` applies at every validation level + - [ ] 07: JSON representation of numbers in uniqueness keys + - [ ] 07: path-stored fields are strings; FORMAT tokens `YYYY`, `MM`, `DD` + - [ ] 10: map iteration order in CEL comprehensions + - [ ] 12: `add` and `remove` address top-level fields only + - [ ] 12: `body_edits` offsets are Unicode scalar values +- [ ] The implementations in the component matrix hold verified claims + against the final release candidate for the profiles of the collection + specification listed for them. Companion profiles (`merge/0.1`, + `move_detection/0.1`, `type_packs/0.1`, `event_action_interop/0.1`, + `runtime/0.2`) are not part of the stable gate. - [ ] `mdbase-rs` and `@callumalpass/mdbase` implement the spec review cleanup and pass the new adapter-target fixtures. - [x] Every canonical schema validates as Draft 2020-12.