Infer a missing seed origin and merge equal-length lists (05A)

05a-data-contracts.md

@@ -395,30 +395,52 @@ The lock records it (see Pack Identity And Portable Provenance). When the target
 exists, is not an intentionally preserved seed target, and the resource declares
 `upgrade_from`, the engine plans exactly one of these, in order:
 
-1. The live bytes equal the desired document: `preserve`.
-2. The live bytes equal a baseline's document: `update`, writing the desired
-   document byte-for-byte. Byte equality proves the origin, whatever the lock
+Two type documents are **equal** when their frontmatter values are equal and
+their Markdown bodies are identical; YAML formatting and key order do not
+matter, because other writers may re-serialize a type file.
+
+1. The live type equals the desired document: `preserve`.
+2. The live type equals a baseline's document: `update`, writing the desired
+   document byte-for-byte. Equality proves the origin, whatever the lock
    records.
 3. The origin is the desired document: `preserve`. The seed was already upgraded
    and has been edited since.
 4. The origin is a baseline: `update` by a three-way merge of that baseline, the
    live type, and the desired type.
-5. Otherwise, because the origin is unknown or is not a listed baseline:
-   `preserve`, with a `reason` stating that no upgrade baseline applies. The
-   type is left as it is; it does not make the pack conflict.
-
-An engine MUST NOT choose a merge baseline any other way. In particular, it MUST
-NOT merge against a baseline the seed is not known to descend from, because the
-differences between that baseline and the seed's actual origin would be applied
-as if they were the user's edits. The assessment reports, for every seed
-`update`, the baseline used as `upgrade_baseline: { digest, version? }`.
+5. Otherwise the origin is unknown (a lock written before origins were
+   recorded, or none) or is not a listed baseline, and the engine **infers** a
+   merge base. It orders the listed baselines by how many frontmatter leaf
+   values differ from the live type's (a key present on one side only counts
+   once; equal-length arrays compare item by item; any other differing value
+   counts once), fewest first and in listed order on a tie. It takes the first
+   baseline whose three-way merge succeeds and whose merged type validates and
+   implements every exact contract version the desired type does, judged over
+   the pack's own contract resources: `update` with that merge, or `preserve`
+   when the merge changes nothing.
+6. Otherwise, when the desired type declares `implements` and the live type
+   already validates and implements every exact contract version the desired
+   type does: `preserve`. The seed already fits the pack.
+7. Otherwise: `preserve`, with a `reason` stating that no upgrade baseline
+   applies. The type is left as it is; it does not make the pack conflict.
+
+In cases 1, 2, 3, and 5 the lock's origin becomes the desired document; in the
+others it is carried forward. An engine MUST NOT choose a merge baseline any
+other way. An inferred base (case 5) may not be the seed's true origin, so the
+differences between them could be applied as if they were the user's edits; the
+conservative merge bounds this, since a seed whose `version` or other setting
+differs from both sides is a competing change and fails. The assessment
+reports, for every seed `update`, the baseline used as
+`upgrade_baseline: { digest, version? }`, so the change is reviewed like any
+other.
 
 A three-way merge is conservative. The type kind and name MUST match. Unchanged publisher
 settings retain live customizations; unchanged live settings accept publisher
 changes. Object settings merge recursively. Contract implementations merge by
 contract ID only when unambiguous, retaining customized field mappings and
 bindings while updating exact version references. Competing changes (including
-ambiguous implementation lists) MUST fail closed. Other arrays are indivisible.
+ambiguous implementation lists) MUST fail closed. Other arrays merge item by
+item when the baseline, live, and desired arrays have the same length, and are
+otherwise indivisible.
 Missing values and explicit null are distinct. Top-level setting removal
 requires separate manual review. The live Markdown body and unrelated YAML
 nodes MUST be preserved. Reformatting a changed YAML node is permitted.

CHANGELOG.md

@@ -195,10 +195,14 @@ Bases, seed type upgrades, and saved-view identification.
   baselines, each a digest-pinned starter the publisher previously shipped
   (`{ digest, document, version? }`), so a collection can upgrade from any
   supported starter, not only the latest. The engine chooses the merge
-  baseline from the seed's origin and never guesses: an unedited seed equal to
-  any baseline is replaced with the exact desired bytes; an edited seed merges
-  against the baseline it descends from; a seed whose origin is unknown or not
-  listed is preserved with a reason rather than merged or made to conflict.
+  baseline from the seed's origin: an unedited seed equal to any baseline is
+  replaced with the exact desired bytes; an edited seed merges against the
+  baseline it descends from. A seed whose origin is unknown or not listed
+  merges against the nearest listed baseline that merges cleanly into a type
+  still implementing the desired contracts; failing that, it is kept, with a
+  reason unless it already implements them, rather than made to conflict.
+  Type documents compare by frontmatter values and body, not bytes, and
+  equal-length arrays merge item by item.
   Competing changes still fail closed, publication is atomic, and the
   assessment reports the baseline used. Engines that do not support the member
   must reject the manifest. Records are never migrated this way.

scripts/type_pack_model.py

@@ -4,7 +4,9 @@
 conformance fixtures exercise: manifest validation of `upgrade_from`, the lock's
 seed `origin_digest`, the ordered choice of an upgrade baseline, and a
 structural three-way merge of type frontmatter that keeps the live body.
-Contract-implementation merging and other engine detail are out of scope.
+Contract validation, contract-implementation merging and other engine detail
+are out of scope: an inferred merge "still implements the desired contracts"
+when its `implements` entries cover the desired type's.
 """
 
 from __future__ import annotations
@@ -195,27 +197,47 @@ def _plan(self, resource: Resource, entry: dict[str, Any] | None) -> Planned:
             if entry is None:
                 return Planned(resource.target, "create", resource.document, resource.digest)
             return Planned(resource.target, "preserve", None, previous_origin)
-        if live == resource.document:
+        if same_type_document(live, resource.document):
             return Planned(resource.target, "preserve", None, resource.digest)
         if not resource.declares_upgrade:
             return Planned(resource.target, "preserve", None, previous_origin)
         for baseline in resource.baselines:
-            if live == baseline.document:
+            if same_type_document(live, baseline.document):
                 return Planned(resource.target, "update", resource.document, resource.digest, baseline=baseline)
         if previous_origin == resource.digest:
             return Planned(resource.target, "preserve", None, previous_origin)
         baseline = next((item for item in resource.baselines if item.digest == previous_origin), None)
         if baseline is None:
-            return Planned(
-                resource.target, "preserve", None, previous_origin,
-                reason=f"{resource.target}: no upgrade baseline applies to this type's origin",
-            )
+            return self._infer(resource, live, previous_origin)
         try:
             merged = merge_documents(baseline.document, live, resource.document)
         except MergeConflict as conflict:
             return Planned(resource.target, "conflict", None, previous_origin, reason=f"{resource.target}: {conflict}")
         return Planned(resource.target, "update", merged, resource.digest, baseline=baseline)
 
+    def _infer(self, resource: Resource, live: bytes, previous_origin: str | None) -> Planned:
+        """No known origin: merge against the nearest listed baseline that merges cleanly."""
+        live_frontmatter, _ = split_document(live)
+        candidates = sorted(
+            resource.baselines,
+            key=lambda baseline: differences(live_frontmatter, split_document(baseline.document)[0]),
+        )
+        for baseline in candidates:
+            try:
+                merged = merge_documents(baseline.document, live, resource.document)
+            except MergeConflict:
+                continue
+            if implements_as_desired(merged, resource.document):
+                if same_type_document(merged, live):
+                    return Planned(resource.target, "preserve", None, resource.digest)
+                return Planned(resource.target, "update", merged, resource.digest, baseline=baseline)
+        if "implements" in split_document(resource.document)[0] and implements_as_desired(live, resource.document):
+            return Planned(resource.target, "preserve", None, previous_origin)
+        return Planned(
+            resource.target, "preserve", None, previous_origin,
+            reason=f"{resource.target}: no upgrade baseline applies to this type's origin",
+        )
+
     def _plan_managed(self, resource: Resource, entry: dict[str, Any] | None, live: bytes | None) -> Planned:
         if live is None:
             return Planned(resource.target, "create", resource.document, None)
@@ -226,6 +248,32 @@ def _plan_managed(self, resource: Resource, entry: dict[str, Any] | None, live:
         return Planned(resource.target, "conflict", None, None, reason=f"{resource.target} changed")
 
 
+def same_type_document(a: bytes, b: bytes) -> bool:
+    """Equal as type documents: the same frontmatter values and the same body."""
+    if a == b:
+        return True
+    try:
+        return split_document(a) == split_document(b)
+    except ValueError:
+        return False
+
+
+def differences(a: Any, b: Any) -> int:
+    """How many frontmatter leaves differ, for choosing an inferred merge base."""
+    if isinstance(a, dict) and isinstance(b, dict):
+        return sum(differences(a[key], b[key]) if key in a and key in b else 1 for key in {*a, *b})
+    if isinstance(a, list) and isinstance(b, list) and len(a) == len(b):
+        return sum(differences(x, y) for x, y in zip(a, b))
+    return int(a != b)
+
+
+def implements_as_desired(candidate: bytes, desired: bytes) -> bool:
+    def implemented(document: bytes) -> set[tuple[str, str]]:
+        entries = split_document(document)[0].get("implements") or []
+        return {(str(entry.get("contract")), str(entry.get("version"))) for entry in entries if isinstance(entry, dict)}
+    return implemented(desired) <= implemented(candidate)
+
+
 def merge_documents(base: bytes, live: bytes, desired: bytes) -> bytes:
     base_frontmatter, _ = split_document(base)
     live_frontmatter, live_body = split_document(live)
@@ -257,4 +305,6 @@ def merge_values(base: Any, live: Any, desired: Any, path: str) -> Any:
             if value is not MISSING:
                 merged[key] = value
         return merged
+    if all(isinstance(value, list) for value in (base, live, desired)) and len(base) == len(live) == len(desired):
+        return [merge_values(b, l, d, f"{path}/{index}") for index, (b, l, d) in enumerate(zip(base, live, desired))]
     raise MergeConflict(f"competing changes at {path or '/'}")

tests/v0.3/type-packs/type-packs.yaml

@@ -338,7 +338,7 @@ groups:
           ---
     tests:
       - id: seed-upgrade-unknown-origin-preserved
-        name: "a user-owned type at a seed target is never merged into a starter"
+        name: "a user-owned type at a seed target that merges cleanly with no starter is preserved"
         operation: apply_type_pack
         input:
           pack: "examples/v0.3/seed-upgrades/v3/mdbase-pack.yaml"
@@ -360,6 +360,65 @@ groups:
         covers:
           - type_packs.seed_type_upgrade
 
+  - name: "an edited seed with no recorded origin merges against the nearest baseline"
+    setup:
+      config: |
+        spec_version: "0.3.0"
+        settings:
+          validation: error
+      files:
+        _types/note.md: |
+          ---
+          kind: mdbase.type
+          name: note
+          version: 1
+          description: A note.
+          match:
+            where:
+              type: note
+          schema:
+            dialect: json-schema-2020-12
+            value:
+              type: object
+              required: [title]
+              properties:
+                title: { type: string }
+                mood: { type: string }
+              additionalProperties: true
+          ---
+          # Note
+
+          My notes, as I keep them.
+    tests:
+      - id: seed-upgrade-unknown-origin-inferred
+        name: "the v1 baseline is nearest, merges cleanly, and keeps the user's edits"
+        operation: apply_type_pack
+        input:
+          pack: "examples/v0.3/seed-upgrades/v3/mdbase-pack.yaml"
+          history:
+            - apply: "examples/v0.3/seed-upgrades/v1/mdbase-pack.yaml"
+        expect:
+          valid: true
+          runs:
+            - valid: true
+              status: upgrade
+              actions: [update]
+          resources:
+            - target: _types/note.md
+              action: update
+              upgrade_baseline_version: 1
+          target_frontmatter:
+            _types/note.md:
+              /version: 3
+              /schema/value/properties/created: { type: string, format: date-time }
+              /schema/value/properties/mood: { type: string }
+          target_body_contains:
+            _types/note.md: ["My notes, as I keep them."]
+          lock_origin:
+            _types/note.md: "examples/v0.3/seed-upgrades/v3/_types/note.md"
+        covers:
+          - type_packs.seed_type_upgrade
+
   - name: "upgrade baselines are validated"
     setup:
       config: |