From 83169beba4a271160f3c6498697455032b7d3a44 Mon Sep 17 00:00:00 2001 From: callumalpass Date: Wed, 4 Nov 2026 17:22:32 +1100 Subject: [PATCH] Specify path-stored fields for 0.3.0-rc.6 A type can now declare that a field's value lives in the record's path, through collection.path.fields and {field:FORMAT} date placeholders, so a task title kept as the filename or a daily note's date is declared once and every tool reads and writes it the same way. Decoded values are persisted values but take no part in type matching, and are never written to frontmatter. A write that changes a stored field moves the record atomically. Adds 53 fixtures in core/path-fields.yaml, the type-file schema change, the concurrent edits model round-trip check, and SN-102. --- 00-overview.md | 5 +- 02-collection-layout.md | 4 +- 03-records-and-frontmatter.md | 13 +- 04-configuration.md | 6 +- 05-type-files.md | 19 +- 07-collection-semantics.md | 193 +++++- 09-lifecycle.md | 5 +- 10-cel-profile.md | 3 +- 12-operations.md | 75 +- 12a-concurrent-edits.md | 4 + 13-migrations-and-compatibility.md | 29 +- 14-conformance.md | 18 +- CHANGELOG.md | 52 ++ notes/SN-102.md | 43 ++ schemas/v0.3/type-file.schema.json | 20 +- scripts/concurrent_edits_model.py | 233 ++++++- tests/v0.3/README.md | 51 +- tests/v0.3/core/path-fields.yaml | 1035 ++++++++++++++++++++++++++++ tests/v0.3/manifest.yaml | 3 + 19 files changed, 1746 insertions(+), 65 deletions(-) create mode 100644 notes/SN-102.md create mode 100644 tests/v0.3/core/path-fields.yaml diff --git a/00-overview.md b/00-overview.md index 6a8f361..2963fd3 100644 --- a/00-overview.md +++ b/00-overview.md @@ -342,7 +342,8 @@ directly comparable without making one product's internal API normative. ## Versioning This specification uses semantic versioning. The current version is **0.3.0**, -in its fifth release candidate (`0.3.0-rc.5`). 0.3.0 is declared stable once +in its fifth release candidate (`0.3.0-rc.5`); the sixth (`0.3.0-rc.6`) is +in draft. 0.3.0 is declared stable once implementations pass the conformance suite. Collections declare their specification version with `spec_version` in `mdbase.yaml`. Tools declare the profiles and versions they implement. @@ -370,7 +371,7 @@ The keywords `MUST`, `MUST NOT`, `SHOULD`, `SHOULD NOT`, and `MAY` are to be interpreted as described in RFC 2119. Draft notes use ordinary prose and are non-normative. A paragraph that begins -with **Provisional (rc.5).** records a choice made where the design input +with **Provisional (rc.5).** or **Provisional (rc.6).** records a choice made where the design input left a detail open. It is normative in the release candidate and may change before 0.3.0 is declared stable. The [0.3.0-rc.5 release notes](./docs/releases/0.3.0-rc.5.md) list every such diff --git a/02-collection-layout.md b/02-collection-layout.md index 79589fb..4e37ba0 100644 --- a/02-collection-layout.md +++ b/02-collection-layout.md @@ -256,7 +256,7 @@ outcome depends on where the path came from: | Path source | Outcome | | --- | --- | | an explicit path supplied by the caller of create or rename | the operation fails with `path_conflict` before any write | -| a path derived from `collection.path.pattern` (Chapter 07) | the record receives the first free suffixed path | +| a path derived from `collection.path.pattern` (Chapter 07) | the record receives the first free suffixed path; when the pattern has path-stored fields, the operation fails with `path_conflict` instead | | two records that already hold equivalent paths, for example after concurrent creates or an engine copying records onto another file system | the earlier-ordered record keeps the path; each later one receives the first free suffixed path | A suffixed path inserts ` (n)`, a space and a decimal integer in parentheses, @@ -274,3 +274,5 @@ records in the same order computes the same paths. A suffixed path is an ordinary path. Applying the rule never edits a record's frontmatter or body, and the record keeps whatever identity its tool tracks. +When the record's type stores fields in its path (Chapter 07), the suffix +becomes part of the decoded value. diff --git a/03-records-and-frontmatter.md b/03-records-and-frontmatter.md index 4ece361..2a7f371 100644 --- a/03-records-and-frontmatter.md +++ b/03-records-and-frontmatter.md @@ -91,11 +91,13 @@ explicit null. ## Persisted And Effective Frontmatter `frontmatter` always means the parsed mapping persisted in the Markdown file. -It does not contain read defaults, computed values, or other derived data. +It does not contain read defaults, computed values, values decoded from the +path, or other derived data. -`effective_frontmatter` is the derived read/query mapping after applying -`collection.read_defaults` and any other read-time computation supported by the -active profile. +`effective_frontmatter` is the derived read/query mapping built from the +record's persisted values, which add the values of path-stored fields decoded +from its path (Chapter 07), after applying `collection.read_defaults` and any +other read-time computation supported by the active profile. These names have fixed meanings in every operation and provider. An operation MUST NOT place effective values in `frontmatter`, place persisted values in @@ -139,7 +141,8 @@ reconstruct `document` from parsed frontmatter and body when exact source is unavailable. Validation of JSON Schema `required` is against the persisted or draft -frontmatter object, not against effective read defaults. +frontmatter object, with the values of path-stored fields (Chapter 07), not +against effective read defaults. ## Body diff --git a/04-configuration.md b/04-configuration.md index eb7d190..8c22791 100644 --- a/04-configuration.md +++ b/04-configuration.md @@ -85,7 +85,7 @@ single record being written. Checks are divided into three tiers: | Tier | Checks | Effect on a write made through an engine | | --- | --- | --- | -| request and safety | invalid requests, path escapes and unsafe paths, `path_conflict` for an explicit path, configuration and type-file errors, `type_conflict`, `type_membership_changed`, lifecycle failures, `if_revision` failures, expression compilation errors, and `collection.unique` rules with `enforce: write` | always rejected, at every validation level | +| request and safety | invalid requests, path escapes and unsafe paths, `path_conflict` for an explicit path or a path with path-stored fields, `path_value_missing` and `path_value_invalid` on a write, configuration and type-file errors, `type_conflict`, `type_membership_changed`, lifecycle failures, `if_revision` failures, expression compilation errors, and `collection.unique` rules with `enforce: write` | always rejected, at every validation level | | single-record | JSON Schema failures, `format_invalid`, non-mapping frontmatter, and data contract view failures | follows the validation level below | | cross-record | `collection.unique` rules with `enforce: report`, `link_not_found`, link `target_type` mismatches, `ambiguous_link`, and `path_collision` | reported, never rejected | @@ -96,6 +96,10 @@ device, or a later edit can change the other records at any time. The only exception is a uniqueness rule that explicitly opts into `enforce: write` (Chapter 07). +`path_value_inconsistent` and `path_field_in_frontmatter` (Chapter 07) are +single-record checks whose severity is always `warning`: they are reported at +levels `warn` and `error` and never reject a write. + Writes that do not go through an engine's write operations are never rejected. That includes edits made by other tools, files that appear through a file system or synchronization tool, and the results of the merge in Chapter 12A. diff --git a/05-type-files.md b/05-type-files.md index 7b66684..f5f81ed 100644 --- a/05-type-files.md +++ b/05-type-files.md @@ -59,15 +59,19 @@ During collection load, an implementation: contract registry Each valid definition then enters the collection's type registry. Diagnostics -for an invalid definition identify the type-file path and the failing section. +for an invalid definition identify the type-file path and the failing section, +with the code of the failing check, or `invalid_type` when no more specific +code applies. For each record, an implementation: 1. parses the record path and raw persisted frontmatter 2. determines its matched types using Chapter 07 -3. validates the raw frontmatter against every matched JSON Schema -4. evaluates every matched type's collection validators -5. constructs effective read values, including compatible read defaults and +3. decodes the values of path-stored fields from the path (Chapter 07) +4. validates the raw frontmatter, with those values, against every matched + JSON Schema +5. evaluates every matched type's collection validators +6. constructs effective read values, including compatible read defaults and projections supported by the implementation Type selection and record validation are separate operations. An explicit type @@ -132,9 +136,10 @@ remain within the collection or installed pack root that owns the schema. Exactly one of `value` and `ref` is required. A type containing both is invalid. -The JSON Schema validates raw persisted frontmatter. Read defaults, -projections, and lifecycle values enter at the stages defined by their own -chapters. +The JSON Schema validates raw persisted frontmatter, together with the values +of path-stored fields decoded from the record's path (Chapter 07). Read +defaults, projections, and lifecycle values enter at the stages defined by +their own chapters. ## mdbase Sections diff --git a/07-collection-semantics.md b/07-collection-semantics.md index 8db4560..20e599e 100644 --- a/07-collection-semantics.md +++ b/07-collection-semantics.md @@ -320,16 +320,34 @@ explicitly. ## Path Policy -Path policy guides create and rename operations: +Path policy derives the paths of new records and can store field values in +the path instead of in frontmatter: ```yaml collection: path: - pattern: "tasks/{id}.md" + pattern: "tasks/{title}.md" + fields: [title] ``` -The portable grammar uses `{field}` placeholders for top-level frontmatter -fields. Values are converted to strings without expression evaluation: a +| Member | Required | Meaning | +| --- | --- | --- | +| `pattern` | when `fields` is present | the path template for derived paths | +| `fields` | no | path-stored fields: top-level fields whose values are stored in the path | +| `x-*` | no | implementation extensions | + +Loading a type whose `collection.path` has any other key reports a `warning` +diagnostic with code `unknown_path_key`, the type name, and `details.key`. The +type still loads and the key has no effect. Runtime-owned path logic and +richer template languages belong under an `x-*` extension. + +### Patterns + +A pattern is literal text and placeholders. A placeholder is `{field}` or +`{field:FORMAT}`, where `field` names a top-level field. `{` and `}` occur only +as placeholder delimiters. + +`{field}` converts the value to a string without expression evaluation: a string is used as written, and a number or boolean uses its JSON representation. A missing or null value produces `path_value_missing`. @@ -348,12 +366,167 @@ field, when it: - contains `/`, `\`, or a NUL character - begins with `.` -A title such as `Q3/Q4 plan` therefore cannot create a subfolder. Generated -paths MUST remain inside the collection root and MUST satisfy the path safety -rules of Chapter 02. When a derived path's path key is already in use, the -record receives the first free suffixed path defined in Chapter 02; the -operation does not fail. Runtime-owned path logic and richer template -languages belong under an `x-*` extension. +A title such as `Q3/Q4 plan` therefore cannot create a subfolder. + +`{field:FORMAT}` writes a date. FORMAT is read from left to right as the +tokens `YYYY` (the four-digit year), `MM` (the two-digit month), and `DD` (the +two-digit day), taken wherever one begins, and literal characters, which MUST +NOT be `{`, `}`, `/`, or `\`. FORMAT contains at least one token. +`{date:DD-MM-YYYY}` writes `2026-10-30` as `30-10-2026`. The value MUST be a +string holding an RFC 3339 `full-date`; any other value fails with +`path_value_invalid` naming the field. `{field:FORMAT}` is valid only for a +field whose top-level property in the type's JSON Schema declares +`format: date`. + +A type whose pattern has a malformed placeholder, or a `{field:FORMAT}` that +breaks these rules, is invalid with an `invalid_type` error naming the type. + +Generated paths MUST remain inside the collection root and MUST satisfy the +path safety rules of Chapter 02. When a derived path's path key is already in +use, the record receives the first free suffixed path defined in Chapter 02, +and the operation does not fail, unless the pattern has path-stored fields +(below). + +### Path-stored fields + +```yaml +collection: + path: + pattern: "Daily/{date:YYYY}/{date:YYYY-MM-DD}.md" + fields: [date] +``` + +Each field in `fields` is a **path-stored field**: its value lives in the +record's path and is never written to frontmatter. A type is invalid, with an +`invalid_type` error naming the type, when `fields` is present and: + +- `pattern` is absent, or a listed field does not occur in `pattern` +- a field is listed twice, or its top-level property in the type's JSON + Schema does not declare `type: string` +- two placeholders occur with no literal character between them and either + names a path-stored field +- a path-stored field occurs both as `{field}` and as `{field:FORMAT}`, or + its `{field:FORMAT}` occurrences do not together contain `YYYY`, `MM`, and + `DD` + +#### Decoding + +The values of a record's path-stored fields are decoded from its +collection-relative path, using the pattern of each matched type that stores +fields: + +1. The pattern matches the whole path. Literal text matches itself exactly, + comparing code points without case folding or normalization. In + `{field:FORMAT}`, `YYYY` matches four ASCII digits, `MM` and `DD` each + match two, and each literal character matches itself. `{field}` matches + one or more characters other than `/`. +2. When the path can match in more than one way, the placeholders are + assigned from left to right, each taking the shortest text with which the + rest of the pattern still matches the rest of the path. +3. A path-stored field written `{field}` has the text its occurrence matched. + If it occurs more than once and the texts differ, it has no value. +4. A path-stored field written `{field:FORMAT}` has the `full-date` formed + from the year, month, and day matched by all its occurrences. If two + occurrences match different values for one component, or the components + do not form a calendar date, it has no value. +5. Text matched by placeholders of other fields is discarded. + +A field left without a value by step 3 or 4 is reported as a `warning` with +code `path_value_inconsistent` and the field. When the path does not match the +pattern, every path-stored field of that type is missing, and nothing is +reported for the path itself; validation reports what a missing value breaks, +such as `required`. Decoding never normalizes the text. + +With `pattern: "notes/{id} {title}.md"` and `fields: [title]`, the path +`notes/A1 Call Bob back.md` decodes `title` as `Call Bob back`: `{id}` takes +the shortest text, `A1`. With `tasks/{title}.md`, `tasks/v1.2 notes.md` +decodes `title` as `v1.2 notes`, because the match is anchored at both ends. +With the daily-note pattern above, `Daily/2026/2026-10-30.md` decodes `date` +as `2026-10-30`, and `Daily/2025/2026-10-30.md` leaves it without a value +and reports `path_value_inconsistent`. + +#### Record values + +A record's **persisted values** are its persisted frontmatter with each +path-stored field set to its decoded value, or removed when it has none. +Outside type matching, every rule that reads a typed record's persisted or raw +frontmatter reads its persisted values: JSON Schema validation, including +`required`; collection validators and uniqueness; links; lifecycle drafts and +`old`; CEL `raw`, `record`, and top-level fields; queries; data contract +views; and display metadata. `effective_frontmatter` is built from the +persisted values. The `frontmatter` member of a record document is still the +mapping parsed from the file and never holds a decoded value (Chapter 03). +Decoded values are not read defaults; a read default applies to a path-stored +field only when it is missing. + +Decoded values never take part in type matching. `match` reads the raw +frontmatter and the path, so a `fields_present` or `where` rule does not see a +path-stored field; the matched types then decide which patterns decode. When +several matched types store one field and decode different values, the record +reports `type_conflict` for the field, and the value of the first matched type +in canonical order is used. + +When the frontmatter also has a key named by a path-stored field, the decoded +value takes its place, or the field is missing when the path does not decode, +and the record reports a `warning` with code `path_field_in_frontmatter` and +the field. No tool moves a file because of such a key. A write through an +engine that sets the field removes the key. + +#### Writing + +A write never persists a path-stored field to frontmatter. Before a value of a +path-stored field written `{field}` is placed in a path, it is normalized: + +1. each character with the Unicode `White_Space` property becomes U+0020 + SPACE; +2. each of `<`, `>`, `:`, `"`, `/`, `\`, `|`, `?`, `*`, `#`, `[`, `]`, and + `^` is removed, and so is each control character (U+0000–U+001F, + U+007F–U+009F) and each invisible or HFS+-ignored character listed in the + portable path rules of Chapter 02; +3. each run of spaces becomes one space; +4. leading and trailing characters that are `.` or a space are removed. + +The normalized value replaces the value that was written: it is the record's +value and the value the operation result reports. Normalizing a normalized +value changes nothing. A value that is not a string, or that is empty after +normalization, fails with `path_value_invalid` naming the field, and so does a +value that gives the derived path a segment the portable path rules of +Chapter 02 forbid, such as the Windows device name `CON` or a segment longer +than 255 bytes. A missing or null value fails with `path_value_missing`. +Values for `{field:FORMAT}` are not normalized; they MUST be a `full-date`. +A write whose new path does not decode to the values it wrote fails with +`path_value_invalid` naming the first such field: with +`pattern: "{title} {id}.md"`, the title `x y` would give `x y A1.md`, which +decodes `title` as `x`. Only values a write sets are normalized, and only a +changed path-stored value moves a record: a write that sets no path-stored +field never moves it and succeeds whether or not its path decodes. + +When a write changes path-stored fields of an existing record, its new path is +the current path with the text matched by each occurrence of each changed +field, as assigned by decoding, replaced by the new value: as written for +`{field}`, formatted for `{field:FORMAT}`. When the current path does not +decode, the new path is derived from the whole pattern and the record's +persisted values. Chapter 12 defines how create, update, and rename use these +rules. In summary: + +- a derived path whose path key belongs to another record fails with + `path_conflict`; it never receives a suffix, which would change the stored + value +- an update that changes a path-stored field moves the record, and can + update the links that pointed to the old path +- a rename sets path-stored values to what its target path decodes, and + fails with `path_value_missing` when the target does not decode +- a file moved outside an engine changes its values by decoding + +The collision rule of Chapter 02 still applies to paths no single write +chose: records created concurrently, records copied onto another file system, +and merged paths (Chapter 12A). Its suffix then becomes part of the decoded +value, such as `Call Bob (2)`. + +**Provisional (rc.6).** Only string fields can be path-stored, and FORMAT +has only the `YYYY`, `MM`, and `DD` tokens. Normalization follows the +filename rules TaskNotes applies to titles, made idempotent: removals happen +before runs of spaces are collapsed and before the ends are trimmed. ## Merge Strategies diff --git a/09-lifecycle.md b/09-lifecycle.md index b820188..a5cf0ee 100644 --- a/09-lifecycle.md +++ b/09-lifecycle.md @@ -126,7 +126,10 @@ For create and update: 8. write the file Lifecycle MUST run before final validation so generated IDs and timestamps can -satisfy required schema fields. Chapters 05 and 12 define membership freezing. +satisfy required schema fields. It also runs before a path is derived from +path-stored fields (Chapter 07): the draft holds the persisted values, +including path-stored values, and a lifecycle action that sets a path-stored +field determines the record's path. Chapters 05 and 12 define membership freezing. Read defaults MUST NOT run as lifecycle policy unless a write operation explicitly asks to materialize them. diff --git a/10-cel-profile.md b/10-cel-profile.md index 15c8003..2a79e2d 100644 --- a/10-cel-profile.md +++ b/10-cel-profile.md @@ -95,7 +95,8 @@ reserved. A frontmatter field with one of those names remains available through ### Query Context In a query, top-level fields and `record` contain effective values. `raw` -contains persisted frontmatter. +contains persisted frontmatter, with the values of path-stored fields decoded +from the record's path (Chapter 07). ```cel !has(raw.status) && record.status == "open" diff --git a/12-operations.md b/12-operations.md index a9d83d9..bcdc671 100644 --- a/12-operations.md +++ b/12-operations.md @@ -65,8 +65,24 @@ Pipeline: An explicit `path` whose path key (Chapter 02) equals that of an existing record fails with `path_conflict`. A path derived from `collection.path.pattern` -never fails for that reason: it receives the first free suffixed path from -Chapter 02, and the result reports the final path. +without path-stored fields never fails for that reason: it receives the first +free suffixed path from Chapter 02, and the result reports the final path. + +When a selected type has path-stored fields (Chapter 07): + +- Without an explicit `path`, the path is derived in step 8 from the values + after lifecycle `on_create`, so a lifecycle action can supply a + path-stored value. Each path-stored value is normalized first. A derived + path whose path key equals that of an existing record fails with + `path_conflict`; it never receives a suffix. +- With an explicit `path`, the path-stored values are decoded from it and + enter the draft before lifecycle. A path that does not decode fails with + `path_value_missing` naming the first such field in `fields` order. A + request or lifecycle value for a path-stored field that differs from the + decoded value after normalization fails with `invalid_request`, the field, + and `details.reason: path_value_mismatch`. +- Path-stored fields are removed from the persisted frontmatter, and the + result reports their values in `effective_frontmatter`. Static JSON Schema defaults MAY be used by editor and create interfaces. Validation-time mutation occurs when the create operation explicitly copies a @@ -108,7 +124,8 @@ Pipeline: replacement or `body_edits` 3. re-match and freeze types when type-affecting fields or path changed 4. apply lifecycle `on_update` -5. verify type membership did not change as a lifecycle side effect +5. derive the new path when a path-stored field changed (below), and verify + type membership did not change as a lifecycle side effect or by the move 6. validate JSON Schema 7. run collection validators, applying the validation tiers of Chapter 04 8. write the changed frontmatter entries with format fidelity (Chapter 12A) @@ -130,6 +147,8 @@ A structured update accepts: document record (Chapter 03) is `invalid_request` - `body_edits` with `body_base`: optional text edits to the body against a known base body, as an alternative to `body`; see below +- `update_refs`: optional, with the meaning and default it has for rename; + it applies when the update moves the record Unsetting a key that is already missing is not an error. A request that names the same field in `patch` and `unset`, names a field inside a key that `patch` @@ -175,6 +194,37 @@ not require `uniqueItems`. A list emptied by `remove` stays as an empty list. **Provisional (rc.5).** `add` and `remove` address top-level fields only, like `patch`, because the merge unit is the top-level field. +### Path-stored fields + +An update that changes the value of a path-stored field (Chapter 07), through +`patch` or a lifecycle `on_update` action, moves the record. A value changes +when, after normalization, it differs from the decoded value: + +- The value is normalized, and the new path is derived as Chapter 07 + defines, after lifecycle has run. +- A new path with the same path key and the same spelling as the current + path is no move. A new path with the same path key and a different + spelling, such as a change of case, respells the path. +- A new path whose path key belongs to another record fails with + `path_conflict`, and nothing is written. +- A new path at which a type that stored the field no longer matches fails + with `type_membership_changed`. +- `unset` of a path-stored field fails with `path_value_missing`, and a null + value fails as Chapter 07 defines. + +The move, the changed values, and, when `update_refs` is enabled, the link +rewrites of a rename to the new path are one atomic change: one record is +moved and changed, and the referring records are rewritten. The move keeps the +record's identity, as a rename does. The result is the record document at its +new `path`, with `references_updated` as for rename when references were +updated. + +A `document` replacement whose frontmatter has a key named by a path-stored +field supplies that field's value: a value equal to the current one after +normalization is dropped from the persisted frontmatter, and a different +value moves the record as above. Links are rewritten only when the request +enables `update_refs`. + ### Body edits `body_edits` changes parts of the body instead of replacing all of it. It is @@ -274,6 +324,12 @@ path key equals that of a different existing record fails with `path_conflict`. A target whose path key equals the source's own path key only changes the path's spelling, such as its case, and is not a conflict. +A rename sets the record's path-stored values (Chapter 07) to what the target +path decodes to. When a type that matches the record at the target path stores +fields and the target does not decode against its pattern, the rename fails +with `path_value_missing` naming the first such field in `fields` order, and +nothing is written. + If reference updating is enabled, the rename updates the links that reference the renamed record: the links that, before the rename, resolve to it under Chapter 08, including a simple link whose filename match was selected by @@ -508,8 +564,9 @@ diagnostics: [] `valid` is false when any error-severity diagnostic applies. Successful `create`, `update`, and `rename` operations MUST return the complete, -authoritative post-write record document defined in Chapter 03. `rename` adds -its rename-specific metadata to that document. +authoritative post-write record document defined in Chapter 03. `rename`, and +an `update` that moves a record, add their rename-specific metadata to that +document. The document MUST be derived from the bytes actually persisted after lifecycle hooks, normalization, validation, and the atomic write complete. Clients and @@ -529,5 +586,11 @@ runtime state, or revisions. After a successful mutation, tools MAY emit watch/runtime events. A tool that reports moves of files edited outside it uses the move detection of Chapter 12A. Events MUST be -delivered after the derived read/query state is consistent. Watch consumers and +delivered after the derived read/query state is consistent. + +A change of path that changes path-stored values, whether by an update, a +rename, or a detected move, is reported as `record_renamed` followed by +`record_modified`, whose `changed_fields` includes each path-stored field whose +value changed (Chapter 14). An update that moves a record is therefore +reported exactly like a rename to the new path combined with a modification. Watch consumers and workflow runtimes may subscribe to the same stream. diff --git a/12a-concurrent-edits.md b/12a-concurrent-edits.md index f7dcc9b..92746e8 100644 --- a/12a-concurrent-edits.md +++ b/12a-concurrent-edits.md @@ -299,6 +299,10 @@ a frontmatter key with the `conflict` strategy: one side's move wins over an unchanged path, and two different moves are a conflict of kind `path`. The merged path then goes through the path collision rule of Chapter 02. +A path-stored field (Chapter 07) is part of the path, not of the frontmatter. +An edit that changes its value is a change of path and merges under this rule, +and the merged record's path-stored values are decoded from the merged path. + ### Validity and lifecycle The merge never checks validity. A merged version is validated when it is diff --git a/13-migrations-and-compatibility.md b/13-migrations-and-compatibility.md index 9c010ff..89828ca 100644 --- a/13-migrations-and-compatibility.md +++ b/13-migrations-and-compatibility.md @@ -3,8 +3,9 @@ ## Migration Philosophy This chapter has two parts. The first lists what changed for v0.3 collections -and engines between the fourth and fifth release candidates. The second -migrates v0.2.x collections into the v0.3 source model. +and engines between the fourth and fifth, and the fifth and sixth, release +candidates. The second migrates v0.2.x collections into the v0.3 source +model. Migration never rewrites a record. @@ -86,6 +87,30 @@ rc.5 suite. The [rc.5 release notes](./docs/releases/0.3.0-rc.5.md) list every conformance test whose expected outcome changed, which is where an rc.4 engine diverges. +## Changes Since rc.5 + +0.3.0-rc.6 (draft) adds path-stored fields (Chapter 07): fields whose values +live in the record's path, such as a task title kept as the filename or a +daily note's date. Collections keep `spec_version: "0.3.0"`. A collection that +does not declare `collection.path.fields` behaves as under rc.5, apart from +the warning below. + +| Area | rc.5 | rc.6 | To keep the rc.5 behavior | +| --- | --- | --- | --- | +| `collection.path` keys other than `pattern` | `runtime`, `template`, `folder`, and `generated_by` accepted and ignored | any key other than `pattern`, `fields`, and `x-*` loads with an `unknown_path_key` warning and is ignored | move runtime-owned path logic under an `x-*` key | +| `{field:FORMAT}` placeholders | not defined | write a `format: date` field with the `YYYY`, `MM`, and `DD` tokens | nothing | +| a value stored only in the filename | applications decode it themselves | `collection.path.fields` decodes it into the record's persisted values | nothing | + +A record whose title is already its filename keeps its file: declaring +`fields: [title]` makes the existing filename the title. A record that also +has `title` in its frontmatter reports `path_field_in_frontmatter` until a +write sets the title, which removes the key. + +An engine moving from rc.5 to rc.6 decodes path-stored fields on read, never +writes them to frontmatter, derives paths from them after lifecycle, moves a +record when an update changes one, and reports `unknown_path_key`, +`path_value_inconsistent`, and `path_field_in_frontmatter`. + ## From v0.2 To v0.3 The rest of this chapter migrates v0.2.x collections into the v0.3 source diff --git a/14-conformance.md b/14-conformance.md index e75003f..70e0a3b 100644 --- a/14-conformance.md +++ b/14-conformance.md @@ -76,7 +76,8 @@ The shared fixtures live in `tests/v0.3/`, and `tests/v0.3/README.md` defines their format. That includes the `merge_records` form, which states a merge case as a base version, two edited versions, and the type declarations, with the exact merged bytes and conflicts as the expected result. Each test added in a -release candidate after rc.4 records it with `since: 0.3.0-rc.5`. +release candidate after rc.4 records that release candidate, such as +`since: 0.3.0-rc.5` or `since: 0.3.0-rc.6`. ### Portable Interoperability Testbed @@ -151,7 +152,8 @@ The v0.3 core codes include `invalid_request`, `duplicate_batch_path`, `projection_shadowed`, `link_not_found`, `link_target_type_mismatch`, `ambiguous_link`, `duplicate_value`, `path_conflict`, `path_collision`, `type_conflict`, `type_membership_changed`, `path_value_missing`, -`path_value_invalid`, `nondeterministic_match`, `invalid_pattern`, +`path_value_invalid`, `path_value_inconsistent`, `path_field_in_frontmatter`, +`unknown_path_key`, `invalid_type`, `nondeterministic_match`, `invalid_pattern`, `schema_ref_forbidden`, `schema_ref_unresolved`, `schema_ref_cycle`, `format_invalid`, `lifecycle_expression_error`, `concurrent_modification`, `invalid_query`, @@ -232,6 +234,8 @@ Collection Semantics implementations MUST: `enforce: write` uniqueness rules - load `collection.merge` declarations and derive default merge strategies - validate portable `collection.path` policies for write-capable tools +- decode path-stored fields from record paths and use the decoded values + wherever persisted values are read, except type matching - expose display metadata as advisory values - compose compatible behavior from multiple matched types - report `type_conflict` for incompatible matched behavior @@ -386,6 +390,10 @@ Core Write implementations MUST: - reject an explicit path that collides under path equivalence with `path_conflict`, give a colliding derived path the first free suffix, and reject invalid path-pattern values with `path_value_invalid` +- never persist path-stored fields to frontmatter, normalize their values, + derive paths from them after lifecycle, reject a colliding derived path + with `path_conflict`, and move a record atomically when an update changes + a path-stored field - enforce `enforce: write` uniqueness rules in a single write order - enforce `if_revision` when the caller supplies it, and never supply it on a caller's behalf @@ -477,8 +485,10 @@ by `record_created`. `record_created` and `record_modified` include current effective frontmatter. `record_renamed` includes `path`, `previous_path`, and current effective frontmatter. `record_modified.changed_fields` contains the top-level raw -frontmatter keys whose persisted values changed; it is empty for a body-only -change. A deleted notification may include the last observed frontmatter. +frontmatter keys whose persisted values changed, and the path-stored fields +(Chapter 07) whose decoded values changed; it is empty for a body-only +change. A change of path that changes path-stored values is reported as +`record_renamed` followed by `record_modified`. A deleted notification may include the last observed frontmatter. Watch implementations MUST: diff --git a/CHANGELOG.md b/CHANGELOG.md index c0c311e..1355ff6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,58 @@ All notable changes to this specification and conformance suite are documented here. +## 0.3.0-rc.6 (draft, untagged) + +The sixth release candidate lets a type store field values in the record's +path, so that a task title kept as the filename or a daily note's date is +declared once, in the type, and every tool reads and writes it the same way. +Collections keep `spec_version: "0.3.0"`. It supersedes the filename-derived +values of [SN-056](./notes/SN-056.md); see [SN-102](./notes/SN-102.md). + +### Path-stored fields + +#### Added + +- `collection.path.fields` lists path-stored fields: top-level string fields + whose values live in the path and are never written to frontmatter + (Chapter 07). +- `{field:FORMAT}` placeholders write a `format: date` field with the tokens + `YYYY`, `MM`, and `DD`, such as `Daily/{date:YYYY}/{date:YYYY-MM-DD}.md`. +- Decoding: the pattern matches the whole path, placeholders take the + shortest text from left to right, and formatted date occurrences combine. + Decoded values are persisted values for validation, uniqueness, queries, + CEL, links, lifecycle, data contract views, and display, but not for type + matching. `effective_frontmatter` holds them; `frontmatter` does not. +- Normalization of stored values before they are placed in a path: + whitespace becomes spaces; `< > : " / \ | ? * # [ ] ^`, control + characters, and the invisible characters of Chapter 02 are removed; space + runs collapse; leading and trailing `.` and spaces are removed. +- Writes (Chapter 12): create derives the path after lifecycle and fails + with `path_conflict` instead of taking a suffix when the path is taken; an + update that changes a stored field moves the record atomically, with + optional `update_refs` and `references_updated`; `unset` of a stored field + and a rename to a path that does not decode fail with + `path_value_missing`. A moving update is reported as `record_renamed` + followed by `record_modified`. +- Diagnostic codes `path_value_inconsistent`, `path_field_in_frontmatter`, + and `unknown_path_key` (warnings), and `invalid_type` for type files that + fail a check without a more specific code. +- `collection.path.fields` and `x-*` keys in + `schemas/v0.3/type-file.schema.json`. +- `tests/v0.3/core/path-fields.yaml`: 52 tests marked `since: 0.3.0-rc.6`, + 24 of them pure `decode_path` and `derive_path` cases executed against + `scripts/concurrent_edits_model.py`. New requirements + `collection_semantics.path_stored_fields` and + `core_write.path_stored_writes`. + +#### Changed + +- Keys of `collection.path` other than `pattern`, `fields`, and `x-*`, + including `runtime`, `template`, `folder`, and `generated_by`, load with an + `unknown_path_key` warning. The type-file schema still accepts them. +- `path_value_missing` and `path_value_invalid` on a write are listed in the + request and safety tier (Chapter 04). + ## 0.3.0-rc.5 (draft, untagged) The fifth release candidate specifies the data semantics that let several diff --git a/notes/SN-102.md b/notes/SN-102.md new file mode 100644 index 0000000..00c5fa3 --- /dev/null +++ b/notes/SN-102.md @@ -0,0 +1,43 @@ +--- +id: SN-102 +title: "Field values stored in the record path" +sections: + - "§7 Path Policy" + - "§12 Operations" +status: resolved +kind: gap +severity: medium +--- + +**Sections:** §7 Path Policy, §12 Operations + +**Issue:** Collections often keep a field's value only in the path: TaskNotes +stores task titles as filenames, and Obsidian daily notes store the date only +in the filename. v0.3 could derive a path from field values +(`collection.path.pattern`) but not read a value back from a path, so every +client decoded and renamed in its own code (TaskNotes through a `titlePolicy` +contract binding and `collection.path` keys the engine ignored). SN-056 asked +for filename-derived values through v0.2 `generated: {from: file.name}`, which +v0.3 does not have, and left parsing, renames, and cycles open. + +**Resolution (0.3.0-rc.6):** `collection.path.fields` declares path-stored +fields. Supersedes SN-056 for filename-derived values. + +1. The pattern is decoded against the record's path: anchored, placeholders + taking the shortest text from left to right, `{field:FORMAT}` date + occurrences combining their `YYYY`, `MM`, and `DD` components. +2. Decoded values are persisted values for validation, queries, CEL, links, + lifecycle, and contracts, but never for type matching, which avoids the + cycle SN-056 noted. +3. Stored fields are never written to frontmatter. Writes normalize the + value, derive the path after lifecycle, fail with `path_conflict` rather + than suffix a taken path, and move the record when an update changes the + value; a rename sets the values from the new path. +4. Other `collection.path` keys load with an `unknown_path_key` warning. + +SN-056's directory-derived values (`file.folder`) and slugs remain out of +scope: a pattern's placeholders never contain `/`, and patterns do not +slugify. + +**Conformance impact:** `tests/v0.3/core/path-fields.yaml`, requirements +`collection_semantics.path_stored_fields` and `core_write.path_stored_writes`. diff --git a/schemas/v0.3/type-file.schema.json b/schemas/v0.3/type-file.schema.json index 7e9f928..b42040e 100644 --- a/schemas/v0.3/type-file.schema.json +++ b/schemas/v0.3/type-file.schema.json @@ -344,12 +344,22 @@ } }, "pathPolicy": { + "description": "Path policy (Chapter 07). Keys other than pattern, fields, and x-* load with an unknown_path_key warning; runtime, template, folder, and generated_by keep their earlier shapes.", "type": "object", "properties": { "pattern": { "type": "string", "minLength": 1 }, + "fields": { + "description": "Path-stored fields: top-level string fields whose values are stored in the path (Chapter 07).", + "type": "array", + "items": { + "type": "string", + "minLength": 1 + }, + "uniqueItems": true + }, "runtime": { "$ref": "#/$defs/identifier" }, @@ -364,7 +374,15 @@ "$ref": "#/$defs/identifier" } }, - "additionalProperties": false + "patternProperties": { + "^x-[A-Za-z][A-Za-z0-9._:-]{0,127}$": { + "$ref": "#/$defs/domainExtension" + } + }, + "dependentRequired": { + "fields": ["pattern"] + }, + "additionalProperties": true }, "mergeStrategies": { "description": "Per-field merge strategies for top-level frontmatter fields (Chapter 07).", diff --git a/scripts/concurrent_edits_model.py b/scripts/concurrent_edits_model.py index 3b0a19e..06d6828 100644 --- a/scripts/concurrent_edits_model.py +++ b/scripts/concurrent_edits_model.py @@ -6,6 +6,7 @@ - path keys, path equivalence, and the collision suffix rule (Chapter 02) - path-pattern derivation and placeholder validation (Chapter 07) +- path-stored field normalization and decoding (Chapter 07, since rc.6) - default and declared merge strategies (Chapter 07) - the three-way record merge and writer format fidelity (Chapter 12A) - move detection pairing (Chapter 12A) @@ -155,33 +156,216 @@ def __init__(self, code: str, field_name: str | None = None): self.field = field_name -def derive_path(pattern: str, frontmatter: dict[str, Any]) -> str: - out = [] +DATE_TOKENS = ("YYYY", "MM", "DD") +FULL_DATE = re.compile(r"^([0-9]{4})-([0-9]{2})-([0-9]{2})$") + +# Chapter 02: characters that are invisible or that HFS+ ignores in names. +INVISIBLE_RANGES = [ + (0x00AD, 0x00AD), (0x034F, 0x034F), (0x115F, 0x1160), (0x17B4, 0x17B5), + (0x180B, 0x180F), (0x200B, 0x200F), (0x202A, 0x202E), (0x2060, 0x206F), + (0x3164, 0x3164), (0xFE00, 0xFE0F), (0xFEFF, 0xFEFF), (0xFFA0, 0xFFA0), + (0xFFF0, 0xFFF8), (0x1BCA0, 0x1BCA3), (0x1D173, 0x1D17A), (0xE0000, 0xE0FFF), +] +# Unicode White_Space property. +WHITE_SPACE = set( + [0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x20, 0x85, 0xA0, 0x1680] + + list(range(0x2000, 0x200B)) + + [0x2028, 0x2029, 0x202F, 0x205F, 0x3000] +) +REMOVED_CHARACTERS = set('<>:"/\\|?*#[]^') +DEVICE_NAMES = {"CON", "PRN", "AUX", "NUL", "CONIN$", "CONOUT$", "CLOCK$"} | { + f"{prefix}{digit}" for prefix in ("COM", "LPT") for digit in "0123456789\u00b9\u00b2\u00b3" +} + + +def parse_pattern(pattern: str) -> list[tuple]: + """Split a path pattern into ("lit", text) and ("ph", field, format_tokens | None) parts. + + format_tokens is a list of "YYYY", "MM", "DD", or ("lit", char) items. + """ + parts: list[tuple] = [] rest = pattern while "{" in rest: open_index = rest.index("{") close_index = rest.index("}", open_index) - out.append(rest[:open_index]) - name = rest[open_index + 1 : close_index] + if open_index: + parts.append(("lit", rest[:open_index])) + body = rest[open_index + 1 : close_index] + name, colon, fmt = body.partition(":") + tokens = None + if colon: + tokens = [] + i = 0 + while i < len(fmt): + for token in DATE_TOKENS: + if fmt.startswith(token, i): + tokens.append(token) + i += len(token) + break + else: + tokens.append(("lit", fmt[i])) + i += 1 + parts.append(("ph", name, tokens)) + rest = rest[close_index + 1 :] + if rest: + parts.append(("lit", rest)) + return parts + + +def normalize_stored_value(value: str) -> str: + """Chapter 07 normalization of a path-stored value written `{field}`.""" + chars = [" " if ord(c) in WHITE_SPACE else c for c in value] + kept = [] + for c in chars: + code = ord(c) + if c in REMOVED_CHARACTERS or code <= 0x1F or 0x7F <= code <= 0x9F: + continue + if any(lo <= code <= hi for lo, hi in INVISIBLE_RANGES): + continue + kept.append(c) + text = re.sub(" +", " ", "".join(kept)) + return text.strip(". ") + + +def is_device_name(segment: str) -> bool: + stem = segment.split(".", 1)[0].rstrip(" ") + return stem.upper() in DEVICE_NAMES + + +def format_date(value: Any, tokens: list, name: str) -> str: + match = FULL_DATE.match(value) if isinstance(value, str) else None + if not match: + raise PathError("path_value_invalid", name) + year, month, day = match.groups() + out = [] + for token in tokens: + out.append({"YYYY": year, "MM": month, "DD": day}[token] if isinstance(token, str) else token[1]) + return "".join(out) + + +def derive_path_and_values( + pattern: str, frontmatter: dict[str, Any], fields: list[str] | None = None +) -> tuple[str, dict[str, Any]]: + """Derive a path; path-stored string values written `{field}` are normalized first.""" + stored = set(fields or []) + values: dict[str, Any] = {} + for name in fields or []: + value = frontmatter.get(name) + if value is None: + raise PathError("path_value_missing", name) + values[name] = value + out = [] + owners: list[set[str]] = [set()] + for part in parse_pattern(pattern): + if part[0] == "lit": + out.append(part[1]) + for _ in range(part[1].count("/")): + owners.append(set()) + continue + _, name, tokens = part value = frontmatter.get(name) if value is None: raise PathError("path_value_missing", name) - if isinstance(value, (list, dict)): - raise PathError("path_value_invalid", name) - if isinstance(value, bool): - text = "true" if value else "false" - elif isinstance(value, str): - text = value - elif isinstance(value, float): - text = es_number(value) + if tokens is not None: + text = format_date(value, tokens, name) + elif name in stored: + if not isinstance(value, str): + raise PathError("path_value_invalid", name) + text = normalize_stored_value(value) + values[name] = text + if text == "": + raise PathError("path_value_invalid", name) else: - text = json.dumps(value) - if text == "" or "/" in text or "\\" in text or "\0" in text or text.startswith("."): - raise PathError("path_value_invalid", name) + if isinstance(value, (list, dict)): + raise PathError("path_value_invalid", name) + if isinstance(value, bool): + text = "true" if value else "false" + elif isinstance(value, str): + text = value + elif isinstance(value, float): + text = es_number(value) + else: + text = json.dumps(value) + if text == "" or "/" in text or "\\" in text or "\0" in text or text.startswith("."): + raise PathError("path_value_invalid", name) + if name in stored: + owners[-1].add(name) out.append(text) - rest = rest[close_index + 1 :] - out.append(rest) - return "".join(out) + path = "".join(out) + for segment, names in zip(path.split("/"), owners): + if names and (is_device_name(segment) or len(segment.encode("utf-8")) > 255): + raise PathError("path_value_invalid", sorted(names, key=list(fields or []).index)[0]) + if fields: + decoded = decode_path(pattern, path, list(fields))["values"] + for name in fields: + if decoded.get(name) != values[name]: + raise PathError("path_value_invalid", name) + return path, values + + +def derive_path(pattern: str, frontmatter: dict[str, Any]) -> str: + return derive_path_and_values(pattern, frontmatter)[0] + + +def decode_path(pattern: str, path: str, fields: list[str]) -> dict[str, Any]: + """Chapter 07 decoding: anchored, placeholders shortest-first left to right.""" + parts = parse_pattern(pattern) + regex = ["^"] + groups: list[tuple[str, list | None]] = [] + for part in parts: + if part[0] == "lit": + regex.append(re.escape(part[1])) + continue + _, name, tokens = part + if tokens is None: + regex.append("([^/]+?)") + else: + inner = [] + for token in tokens: + if isinstance(token, str): + inner.append("([0-9]{%d})" % len(token)) + else: + inner.append(re.escape(token[1])) + regex.append("".join(inner)) + groups.append((name, tokens)) + match = re.match("".join(regex) + "$", path) + if not match: + return {"matched": False, "values": {}, "issues": []} + captured = iter(match.groups()) + texts: dict[str, list[str]] = {} + components: dict[str, dict[str, set[str]]] = {} + for name, tokens in groups: + if tokens is None: + texts.setdefault(name, []).append(next(captured)) + else: + for token in tokens: + if isinstance(token, str): + components.setdefault(name, {}).setdefault(token, set()).add(next(captured)) + values: dict[str, Any] = {} + issues = [] + for name in fields: + if name in texts: + if len(set(texts[name])) == 1: + values[name] = texts[name][0] + else: + issues.append({"code": "path_value_inconsistent", "field": name}) + continue + parts_seen = components.get(name, {}) + if all(len(parts_seen.get(token, ())) == 1 for token in DATE_TOKENS): + year, month, day = (next(iter(parts_seen[token])) for token in DATE_TOKENS) + if valid_calendar_date(int(year), int(month), int(day)): + values[name] = f"{year}-{month}-{day}" + continue + issues.append({"code": "path_value_inconsistent", "field": name}) + return {"matched": True, "values": values, "issues": issues} + + +def valid_calendar_date(year: int, month: int, day: int) -> bool: + if not 1 <= month <= 12 or day < 1: + return False + leap = year % 4 == 0 and (year % 100 != 0 or year % 400 == 0) + days = [31, 29 if leap else 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31][month - 1] + return day <= days # --------------------------------------------------------------- equality @@ -980,6 +1164,7 @@ def rebase_body_edits(base: str | None, current: str, base_digest: str, edits: l "path_equivalence", "allocate_path", "derive_path", + "decode_path", "detect_moves", "regex_match", "apply_body_edits", @@ -1041,13 +1226,23 @@ def run_fixture(test: dict[str, Any], setup: dict[str, Any]) -> None: if operation == "derive_path": try: - path = derive_path(data["pattern"], data.get("frontmatter") or {}) + path, values = derive_path_and_values(data["pattern"], data.get("frontmatter") or {}, data.get("fields")) except PathError as exc: if "error" not in expect or expect["error"].get("code") != exc.code or expect["error"].get("field") != exc.field: raise AssertionError(f"unexpected {exc.code} for {exc.field}") from exc return if expect.get("path") != path: raise AssertionError(f"path {path!r} != expected {expect.get('path')!r}") + if "values" in expect and expect["values"] != values: + raise AssertionError(f"values {values!r} != expected {expect['values']!r}") + return + + if operation == "decode_path": + result = decode_path(data["pattern"], data["path"], data["fields"]) + for key in ("matched", "values", "issues"): + wanted = expect.get(key, {"matched": True, "values": {}, "issues": []}[key]) + if result[key] != wanted: + raise AssertionError(f"{key} {result[key]!r} != expected {wanted!r}") return if operation == "detect_moves": diff --git a/tests/v0.3/README.md b/tests/v0.3/README.md index 5ca2f56..f70cc98 100644 --- a/tests/v0.3/README.md +++ b/tests/v0.3/README.md @@ -118,6 +118,7 @@ Future v0.3 adapters should support these operations: - `merge_records`, `merge_strategies`, `path_equivalence`, `allocate_path`, `derive_path`, `detect_moves`, `regex_match`, and `apply_body_edits` (since rc.5; see below) +- `decode_path` (since rc.6; see below) ### Type-pack history @@ -157,11 +158,13 @@ are tracked in [release/v0.3.0.md](../../release/v0.3.0.md). ## Release-candidate markers -Tests added after 0.3.0-rc.4 carry `since: 0.3.0-rc.5`. Existing tests whose -expected outcome changed carry `changed: 0.3.0-rc.5`. An engine that conforms -to rc.4 passes every test without either marker; the release notes in -`docs/releases/0.3.0-rc.5.md` list the changed tests. Requirements added in -rc.5 are marked with a `# since 0.3.0-rc.5` comment in `manifest.yaml`. +Tests added after 0.3.0-rc.4 carry `since: 0.3.0-rc.5`, and tests added +after 0.3.0-rc.5 carry `since: 0.3.0-rc.6`. Existing tests whose expected +outcome changed carry `changed:` with the release candidate that changed it. +An engine that conforms to rc.4 passes every test without either marker; the +release notes in `docs/releases/0.3.0-rc.5.md` list the changed tests. +Requirements added in a release candidate are marked with a +`# since 0.3.0-rc.N` comment in `manifest.yaml`. ## Concurrent-edit, regex, and body-edit operations (since rc.5) @@ -294,3 +297,41 @@ expect: It models the body part of an `update` with `body_edits` (Chapter 12). +## Path-stored field operations (since rc.6) + +`core/path-fields.yaml` covers `collection.path.fields` (Chapter 07). Its pure +cases use `decode_path` and `derive_path`, which `scripts/check_v03_tests.py` +runs against the executable model; the rest use `load_types`, `read`, +`validate`, `query`, `create`, `update`, and `rename`. + +### `decode_path` + +```yaml +operation: decode_path +input: + pattern: "notes/{id} {title}.md" + path: "notes/A1 Call Bob back.md" + fields: [title] # the path-stored fields +expect: + matched: true # whether the pattern matches the whole path + values: { title: "Call Bob back" } # every stored field that has a value + issues: [] # every issue, e.g. { code: path_value_inconsistent, field: date } +``` + +A path that does not match gives `matched: false`, `values: {}`, and +`issues: []`. + +### `derive_path` with stored fields + +`input.fields` names the path-stored fields; their values are normalized before +they are placed in the path, and `expect.values` lists the normalized values. +Without `input.fields`, `derive_path` behaves as in rc.5. Both forms accept +`{field:FORMAT}` placeholders. + +### Fixed clock and update options + +`create` and `update` may give `input.now`, an RFC 3339 timestamp: the +operation's captured instant, which lifecycle `now` and `today` read. The +group's `settings.timezone` gives the operation timezone. `update` may give +`input.update_refs`; its result then has `references_updated`, in the format +of `rename`. diff --git a/tests/v0.3/core/path-fields.yaml b/tests/v0.3/core/path-fields.yaml new file mode 100644 index 0000000..bcc7711 --- /dev/null +++ b/tests/v0.3/core/path-fields.yaml @@ -0,0 +1,1035 @@ +# Path-stored fields (Chapter 07, Path Policy; Chapter 12): fields whose +# values live in the record's path instead of its frontmatter. The +# `decode_path` and `derive_path` cases are pure functions that +# scripts/check_v03_tests.py runs against scripts/concurrent_edits_model.py; +# the other cases need an adapter. +name: "path-stored fields" +spec_version: "0.3.0" +fixture_set: core_collection +category: paths +spec_ref: "v0.3/07, v0.3/12" + +groups: + - name: "decoding" + tests: + - name: "a stored field takes the text its placeholder matched" + id: path_fields.decode_simple + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: decode_path + input: + pattern: "tasks/{title}.md" + path: "tasks/Call Bob.md" + fields: [title] + expect: + matched: true + values: { title: "Call Bob" } + issues: [] + + - name: "the match is anchored, so a dot inside the value is kept" + id: path_fields.decode_anchored + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: decode_path + input: + pattern: "tasks/{title}.md" + path: "tasks/v1.2 notes.md" + fields: [title] + expect: + matched: true + values: { title: "v1.2 notes" } + issues: [] + + - name: "earlier placeholders take the shortest text, later ones the rest" + id: path_fields.decode_shortest_match + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: decode_path + input: + pattern: "notes/{id} {title}.md" + path: "notes/A1 Call Bob back.md" + fields: [title] + expect: + matched: true + values: { title: "Call Bob back" } + issues: [] + + - name: "a non-stored placeholder's text is discarded" + id: path_fields.decode_unstored_discarded + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: decode_path + input: + pattern: "notes/{id} {title}.md" + path: "notes/A1 B2 C3.md" + fields: [id] + expect: + matched: true + values: { id: A1 } + issues: [] + + - name: "formatted date occurrences combine into one full-date" + id: path_fields.decode_date_nested + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: decode_path + input: + pattern: "Daily/{date:YYYY}/{date:MM}/{date:YYYY-MM-DD}.md" + path: "Daily/2026/10/2026-10-30.md" + fields: [date] + expect: + matched: true + values: { date: "2026-10-30" } + issues: [] + + - name: "a format can reorder the components" + id: path_fields.decode_date_reordered + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: decode_path + input: + pattern: "Daily/{date:DD-MM-YYYY}.md" + path: "Daily/30-10-2026.md" + fields: [date] + expect: + matched: true + values: { date: "2026-10-30" } + issues: [] + + - name: "occurrences that disagree leave the field without a value" + id: path_fields.decode_date_inconsistent + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: decode_path + input: + pattern: "Daily/{date:YYYY}/{date:MM}/{date:YYYY-MM-DD}.md" + path: "Daily/2025/10/2026-10-30.md" + fields: [date] + expect: + matched: true + values: {} + issues: + - { code: path_value_inconsistent, field: date } + + - name: "components that are not a calendar date are inconsistent" + id: path_fields.decode_date_not_calendar + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: decode_path + input: + pattern: "Daily/{date:YYYY-MM-DD}.md" + path: "Daily/2026-02-30.md" + fields: [date] + expect: + matched: true + values: {} + issues: + - { code: path_value_inconsistent, field: date } + + - name: "a token matches exactly its number of digits" + id: path_fields.decode_date_digit_count + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: decode_path + input: + pattern: "Daily/{date:DD-MM-YYYY}.md" + path: "Daily/30-10-26.md" + fields: [date] + expect: + matched: false + values: {} + issues: [] + + - name: "repeated plain occurrences that differ are inconsistent" + id: path_fields.decode_repeated_inconsistent + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: decode_path + input: + pattern: "projects/{name}/{name}.md" + path: "projects/alpha/beta.md" + fields: [name] + expect: + matched: true + values: {} + issues: + - { code: path_value_inconsistent, field: name } + + - name: "a path that does not match leaves stored fields missing" + id: path_fields.decode_no_match + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: decode_path + input: + pattern: "tasks/{title}.md" + path: "tasks/archive/Call Bob.md" + fields: [title] + expect: + matched: false + values: {} + issues: [] + + - name: "decoding compares literal text by code point" + id: path_fields.decode_case_sensitive_literal + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: decode_path + input: + pattern: "tasks/{title}.md" + path: "Tasks/Call Bob.md" + fields: [title] + expect: + matched: false + values: {} + issues: [] + + - name: "normalization and derivation" + tests: + - name: "whitespace runs collapse and the ends are trimmed" + id: path_fields.normalize_whitespace + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: derive_path + input: + pattern: "tasks/{title}.md" + fields: [title] + frontmatter: + title: " Call \t Bob\nback " + expect: + path: "tasks/Call Bob back.md" + values: { title: "Call Bob back" } + + - name: "forbidden characters are removed, so a slash cannot create a folder" + id: path_fields.normalize_removed_characters + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: derive_path + input: + pattern: "tasks/{title}.md" + fields: [title] + frontmatter: + title: "Q3/Q4: plan?" + expect: + path: "tasks/Q3Q4 plan.md" + values: { title: "Q3Q4 plan" } + + - name: "every listed character is removed before spaces collapse" + id: path_fields.normalize_full_list + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: derive_path + input: + pattern: "tasks/{title}.md" + fields: [title] + frontmatter: + title: "#tag [link] ^block | \"q\" * a\\b" + expect: + path: "tasks/tag link block x q ab.md" + values: { title: "tag link block x q ab" } + + - name: "control and invisible characters are removed" + id: path_fields.normalize_control_invisible + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: derive_path + input: + pattern: "tasks/{title}.md" + fields: [title] + frontmatter: + title: "a\x07b ​ c­" + expect: + path: "tasks/ab c.md" + values: { title: "ab c" } + + - name: "leading and trailing dots and spaces are removed" + id: path_fields.normalize_dots + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: derive_path + input: + pattern: "tasks/{title}.md" + fields: [title] + frontmatter: + title: " . .hidden draft. . " + expect: + path: "tasks/hidden draft.md" + values: { title: "hidden draft" } + + - name: "a value that would not decode back is invalid" + id: path_fields.derive_round_trip + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: derive_path + input: + pattern: "{title} {id}.md" + fields: [title] + frontmatter: + title: "x y" + id: A1 + expect: + error: + code: path_value_invalid + field: title + + - name: "a value empty after normalization is invalid" + id: path_fields.normalize_empty + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: derive_path + input: + pattern: "tasks/{title}.md" + fields: [title] + frontmatter: + title: " ?:* .. " + expect: + error: + code: path_value_invalid + field: title + + - name: "a Windows device name is invalid" + id: path_fields.normalize_device_name + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: derive_path + input: + pattern: "tasks/{title}.md" + fields: [title] + frontmatter: + title: "con" + expect: + error: + code: path_value_invalid + field: title + + - name: "a stored value that is not a string is invalid" + id: path_fields.normalize_not_string + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: derive_path + input: + pattern: "tasks/{title}.md" + fields: [title] + frontmatter: + title: 42 + expect: + error: + code: path_value_invalid + field: title + + - name: "a missing stored value is path_value_missing" + id: path_fields.normalize_missing + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: derive_path + input: + pattern: "tasks/{title}.md" + fields: [title] + frontmatter: {} + expect: + error: + code: path_value_missing + field: title + + - name: "a formatted placeholder writes each occurrence's components" + id: path_fields.derive_date_format + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: derive_path + input: + pattern: "Daily/{date:YYYY}/{date:DD-MM-YYYY}.md" + fields: [date] + frontmatter: + date: "2026-10-30" + expect: + path: "Daily/2026/30-10-2026.md" + values: { date: "2026-10-30" } + + - name: "a formatted placeholder needs a full-date" + id: path_fields.derive_date_invalid + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: derive_path + input: + pattern: "Daily/{date:YYYY-MM-DD}.md" + fields: [date] + frontmatter: + date: "30/10/2026" + expect: + error: + code: path_value_invalid + field: date + + - name: "a format works for a field that is not stored" + id: path_fields.derive_date_unstored + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: derive_path + input: + pattern: "meetings/{date:YYYY}/{title}.md" + frontmatter: + date: "2026-10-30" + title: Standup + expect: + path: "meetings/2026/Standup.md" + + - name: "type loading" + setup: + config: | + spec_version: "0.3.0" + types: + legacy.md: | + --- + kind: mdbase.type + name: legacy + version: 1 + match: + path_glob: "legacy/*.md" + schema: + dialect: json-schema-2020-12 + value: + type: object + properties: + title: { type: string } + collection: + path: + pattern: "legacy/{title}.md" + fields: [title] + runtime: tasknotes + x-tasknotes: + template: "{{title}}" + --- + tests: + - name: "a key other than pattern, fields, or x-* loads with a warning" + id: path_fields.unknown_path_key + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: load_types + input: {} + expect: + valid: true + diagnostics: + - code: unknown_path_key + severity: warning + type: legacy + details: { key: runtime } + + - name: "invalid path-stored declarations" + setup: + config: | + spec_version: "0.3.0" + types: + adjacent.md: | + --- + kind: mdbase.type + name: adjacent + version: 1 + schema: + dialect: json-schema-2020-12 + value: + type: object + properties: + id: { type: string } + title: { type: string } + collection: + path: + pattern: "notes/{id}{title}.md" + fields: [title] + --- + not-string.md: | + --- + kind: mdbase.type + name: not_string + version: 1 + schema: + dialect: json-schema-2020-12 + value: + type: object + properties: + rank: { type: integer } + collection: + path: + pattern: "ranks/{rank}.md" + fields: [rank] + --- + absent-field.md: | + --- + kind: mdbase.type + name: absent_field + version: 1 + schema: + dialect: json-schema-2020-12 + value: + type: object + properties: + id: { type: string } + title: { type: string } + collection: + path: + pattern: "items/{id}.md" + fields: [title] + --- + format-not-date.md: | + --- + kind: mdbase.type + name: format_not_date + version: 1 + schema: + dialect: json-schema-2020-12 + value: + type: object + properties: + day: { type: string } + collection: + path: + pattern: "days/{day:YYYY-MM-DD}.md" + fields: [day] + --- + partial-date.md: | + --- + kind: mdbase.type + name: partial_date + version: 1 + schema: + dialect: json-schema-2020-12 + value: + type: object + properties: + month: { type: string, format: date } + collection: + path: + pattern: "months/{month:YYYY-MM}.md" + fields: [month] + --- + tests: + - name: "invalid declarations make their types invalid" + id: path_fields.invalid_declarations + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: load_types + input: {} + expect: + valid: false + diagnostics: + - { code: invalid_type, severity: error, type: adjacent } + - { code: invalid_type, severity: error, type: not_string } + - { code: invalid_type, severity: error, type: absent_field } + - { code: invalid_type, severity: error, type: format_not_date } + - { code: invalid_type, severity: error, type: partial_date } + + - name: "title stored in the filename" + setup: + config: | + spec_version: "0.3.0" + settings: + validation: error + types: + task.md: | + --- + kind: mdbase.type + name: task + version: 1 + match: + path_glob: "tasks/**/*.md" + schema: + dialect: json-schema-2020-12 + value: + type: object + required: [title] + properties: + title: { type: string, minLength: 1 } + status: { enum: [open, done] } + collection: + path: + pattern: "tasks/{title}.md" + fields: [title] + --- + files: + tasks/Call Bob.md: | + --- + status: open + --- + Ring before noon. + tasks/Ring Alice.md: | + --- + status: open + --- + tasks/Dup.md: | + --- + title: Other + status: open + --- + tasks/sub/orphan.md: | + --- + status: open + --- + notes/plan.md: | + --- + ref: "[[Call Bob]]" + --- + tests: + - name: "a read reports the decoded value in effective frontmatter only" + id: path_fields.read_decoded + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: read + input: + path: "tasks/Call Bob.md" + expect: + valid: true + types: [task] + frontmatter: { status: open } + frontmatter_not_contains: [title] + effective_frontmatter: { title: "Call Bob", status: open } + diagnostics: [] + + - name: "decoded values are visible to queries through raw" + id: path_fields.query_raw + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: query + input: + types: [task] + where: 'raw.title == "Call Bob"' + expect: + valid: true + results: + - path: "tasks/Call Bob.md" + + - name: "a path that does not decode leaves a required stored field missing" + id: path_fields.required_validation + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: validate + input: + path: "tasks/sub/orphan.md" + expect: + valid: false + issues: + - { code: schema_required, severity: error, field: title } + + - name: "a frontmatter key named by a stored field loses to the path" + id: path_fields.frontmatter_duplicate + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: read + input: + path: "tasks/Dup.md" + expect: + valid: true + frontmatter: { title: Other } + effective_frontmatter: { title: Dup } + diagnostics: + - { code: path_field_in_frontmatter, severity: warning, field: title } + + - name: "create derives the path and never persists the stored field" + id: path_fields.create_not_persisted + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: create + input: + type: task + frontmatter: { title: " Q3/Q4: plan? ", status: open } + expect: + valid: true + path: "tasks/Q3Q4 plan.md" + frontmatter: { status: open } + frontmatter_not_contains: [title] + effective_frontmatter: { title: "Q3Q4 plan" } + + - name: "a derived path taken by another record fails instead of taking a suffix" + id: path_fields.create_collision + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: create + input: + type: task + frontmatter: { title: "call bob", status: open } + expect: + valid: false + error: + code: path_conflict + + - name: "an explicit path supplies the stored value" + id: path_fields.create_explicit_path + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: create + input: + type: task + path: "tasks/Email Carol.md" + frontmatter: { status: open } + expect: + valid: true + path: "tasks/Email Carol.md" + frontmatter_not_contains: [title] + effective_frontmatter: { title: "Email Carol" } + + - name: "an explicit path that disagrees with the requested value is rejected" + id: path_fields.create_explicit_mismatch + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: create + input: + type: task + path: "tasks/Email Carol.md" + frontmatter: { title: "Email Dave", status: open } + expect: + valid: false + error: + code: invalid_request + field: title + details: { reason: path_value_mismatch } + + - name: "patching a stored field moves the record and rewrites its links" + id: path_fields.update_moves + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: update + input: + path: "tasks/Call Bob.md" + patch: { title: "Call Robert" } + update_refs: true + expect: + valid: true + path: "tasks/Call Robert.md" + frontmatter: { status: open } + frontmatter_not_contains: [title] + effective_frontmatter: { title: "Call Robert", status: open } + body_contains: "Ring before noon." + references_updated: + - path: "notes/plan.md" + field: ref + old_value: "[[Call Bob]]" + new_value: "[[Call Robert]]" + verify_after: + operation: read + input: + path: "tasks/Call Robert.md" + expect: + valid: true + effective_frontmatter: { title: "Call Robert" } + + - name: "a case-only change respells the path" + id: path_fields.update_respells + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: update + input: + path: "tasks/Call Bob.md" + patch: { title: "call bob" } + expect: + valid: true + path: "tasks/call bob.md" + effective_frontmatter: { title: "call bob" } + + - name: "the patched value is normalized" + id: path_fields.update_normalizes + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: update + input: + path: "tasks/Call Bob.md" + patch: { title: "Call Bob?" } + expect: + valid: true + path: "tasks/Call Bob.md" + effective_frontmatter: { title: "Call Bob" } + + - name: "an update onto another record's path fails" + id: path_fields.update_collision + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: update + input: + path: "tasks/Call Bob.md" + patch: { title: "ring alice" } + expect: + valid: false + error: + code: path_conflict + verify_after: + operation: read + input: + path: "tasks/Call Bob.md" + expect: + valid: true + effective_frontmatter: { title: "Call Bob" } + + - name: "unsetting a stored field is path_value_missing" + id: path_fields.update_unset + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: update + input: + path: "tasks/Call Bob.md" + unset: [title] + expect: + valid: false + error: + code: path_value_missing + field: title + + - name: "a record whose path does not decode moves to the derived path" + id: path_fields.update_from_undecodable + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: update + input: + path: "tasks/sub/orphan.md" + patch: { title: Orphan } + expect: + valid: true + path: "tasks/Orphan.md" + frontmatter_not_contains: [title] + effective_frontmatter: { title: Orphan } + + - name: "setting the field removes a duplicate frontmatter key" + id: path_fields.update_removes_duplicate_key + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: update + input: + path: "tasks/Dup.md" + patch: { title: Dedup } + expect: + valid: true + path: "tasks/Dedup.md" + frontmatter: { status: open } + frontmatter_not_contains: [title] + effective_frontmatter: { title: Dedup } + diagnostics: [] + + - name: "a document replacement with an equal stored key drops the key" + id: path_fields.document_equal_value + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: update + input: + path: "tasks/Call Bob.md" + document: | + --- + title: Call Bob + status: done + --- + Ring before noon. + expect: + valid: true + path: "tasks/Call Bob.md" + frontmatter: { status: done } + frontmatter_not_contains: [title] + + - name: "a document replacement with a different stored key moves the record" + id: path_fields.document_moves + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: update + input: + path: "tasks/Call Bob.md" + document: | + --- + title: Call Robert + status: open + --- + Ring before noon. + expect: + valid: true + path: "tasks/Call Robert.md" + frontmatter_not_contains: [title] + effective_frontmatter: { title: "Call Robert" } + + - name: "a rename sets the stored value from the target path" + id: path_fields.rename_decodes + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: rename + input: + from: "tasks/Call Bob.md" + to: "tasks/Phone Bob.md" + expect: + valid: true + path: "tasks/Phone Bob.md" + frontmatter_not_contains: [title] + effective_frontmatter: { title: "Phone Bob" } + + - name: "a rename to a path that does not decode fails" + id: path_fields.rename_not_decodable + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: rename + input: + from: "tasks/Call Bob.md" + to: "tasks/archive/Call Bob.md" + expect: + valid: false + error: + code: path_value_missing + field: title + + - name: "id and title in the filename" + setup: + config: | + spec_version: "0.3.0" + types: + zettel.md: | + --- + kind: mdbase.type + name: zettel + version: 1 + match: + path_glob: "zettel/*.md" + schema: + dialect: json-schema-2020-12 + value: + type: object + required: [title] + properties: + id: { type: string } + title: { type: string } + collection: + path: + pattern: "zettel/{id} {title}.md" + fields: [title] + --- + files: + zettel/A1 Old idea.md: | + --- + id: B2 + --- + tests: + - name: "create fills both placeholders and stores only the title" + id: path_fields.zettel_create + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: create + input: + type: zettel + frontmatter: { id: "202610301200", title: First note } + expect: + valid: true + path: "zettel/202610301200 First note.md" + frontmatter: { id: "202610301200" } + frontmatter_not_contains: [title] + + - name: "an update replaces only the stored field's span" + id: path_fields.zettel_update_span + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: update + input: + path: "zettel/A1 Old idea.md" + patch: { title: New idea } + expect: + valid: true + path: "zettel/A1 New idea.md" + frontmatter: { id: B2 } + effective_frontmatter: { id: B2, title: New idea } + + - name: "daily notes" + setup: + config: | + spec_version: "0.3.0" + settings: + timezone: UTC + types: + daily.md: | + --- + kind: mdbase.type + name: daily + version: 1 + match: + path_glob: "Daily/**/*.md" + schema: + dialect: json-schema-2020-12 + value: + type: object + required: [date] + properties: + date: { type: string, format: date } + mood: { type: string } + collection: + path: + pattern: "Daily/{date:YYYY}/{date:YYYY-MM-DD}.md" + fields: [date] + lifecycle: + on_create: + - if: '!has(raw.date)' + set: + date: { today: true } + --- + files: + Daily/2026/2026-10-29.md: | + --- + mood: calm + --- + Daily/2025/2026-10-28.md: | + --- + mood: odd + --- + tests: + - name: "create derives the path after lifecycle supplies the date" + id: path_fields.create_after_lifecycle + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: create + input: + type: daily + now: "2026-10-30T09:00:00Z" + frontmatter: { mood: bright } + expect: + valid: true + path: "Daily/2026/2026-10-30.md" + frontmatter: { mood: bright } + frontmatter_not_contains: [date] + effective_frontmatter: { date: "2026-10-30", mood: bright } + + - name: "a daily note that already exists is a path conflict" + id: path_fields.create_date_collision + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: create + input: + type: daily + frontmatter: { date: "2026-10-29" } + expect: + valid: false + error: + code: path_conflict + + - name: "a read decodes the date from both occurrences" + id: path_fields.read_date + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: read + input: + path: "Daily/2026/2026-10-29.md" + expect: + valid: true + effective_frontmatter: { date: "2026-10-29", mood: calm } + diagnostics: [] + + - name: "disagreeing occurrences report path_value_inconsistent" + id: path_fields.read_date_inconsistent + since: 0.3.0-rc.6 + covers: [collection_semantics.path_stored_fields] + operation: validate + input: + path: "Daily/2025/2026-10-28.md" + expect: + valid: false + issues: + - { code: path_value_inconsistent, severity: warning, field: date } + - { code: schema_required, severity: error, field: date } + + - name: "an update of the date moves the note into the right folder" + id: path_fields.update_date_moves + since: 0.3.0-rc.6 + covers: [core_write.path_stored_writes] + operation: update + input: + path: "Daily/2026/2026-10-29.md" + patch: { date: "2025-12-31" } + expect: + valid: true + path: "Daily/2025/2025-12-31.md" + frontmatter: { mood: calm } + frontmatter_not_contains: [date] diff --git a/tests/v0.3/manifest.yaml b/tests/v0.3/manifest.yaml index 41689c3..715fa96 100644 --- a/tests/v0.3/manifest.yaml +++ b/tests/v0.3/manifest.yaml @@ -47,6 +47,7 @@ claim_profiles: - unique_scope # since 0.3.0-rc.5 - cross_record_reporting # since 0.3.0-rc.5 - merge_declarations # since 0.3.0-rc.5 + - path_stored_fields # since 0.3.0-rc.6 - id: data_contracts status: coverage_complete requires: [core_read] @@ -118,6 +119,7 @@ claim_profiles: - path_pattern_values # since 0.3.0-rc.5 - rename_references # since 0.3.0-rc.5 - body_edits # since 0.3.0-rc.5 + - path_stored_writes # since 0.3.0-rc.6 - id: merge # since 0.3.0-rc.5 status: coverage_complete requires: [core_write] @@ -226,6 +228,7 @@ fixture_sets: - core/validation-tiers.yaml - core/list-ops-and-fidelity.yaml - core/paths.yaml + - core/path-fields.yaml - core/link-ambiguity.yaml - core/rename-references.yaml - core/body-edits-update.yaml