Skip to content

memory: an imported retraction stays retracted on the LadybugDB backend - #123

Merged
Shashankss1205 merged 1 commit into
mainfrom
fix/issue-110-ladybug-superseded-by
Sep 26, 2026
Merged

Shashankss1205 merged 1 commit into
mainfrom
fix/issue-110-ladybug-superseded-by

Conversation

@Shashankss1205

Copy link
Copy Markdown
Collaborator

Closes #110, taking option 3 of the three the issue costed: persist superseded_by as a column and keep the edge derived from it.

The issue said the choice was a persistence-format decision and should be the maintainer's. I've implemented the option the issue recommends and made the decision explicit here so it can be rejected cheaply — the alternatives are one revert away.

The defect

superseded_by was reconstructed from the SUPERSEDED_BY edge, and add() cannot create an edge towards a claim the store does not have yet. In the natural import order it does not have it: all_claims() is oldest-first, so a superseded claim arrives before the claim that superseded it.

The edge was silently skipped while superseded_at was written, so one row said two things:

MemoryStore  superseded_by=b     is_current=False  current(svc)=['b']
SQLite       superseded_by=b     is_current=False  current(svc)=['b']
Ladybug      superseded_by=None  is_current=True   current(svc)=['a', 'b']   <-- both sides

current() is supposed to answer "what is true now" and answered alice and bob. supersede() was never affected — it writes the edge itself, which is exactly why every existing test that went through it passed, including the backend-comparison one.

Why option 3, and what the others cost

  • 1, fail closed — breaks replaying all_claims() in its own returned order, which is the order it hands you.
  • 2, stub target node — the stub reads back as a ValidationError and has no seq, which is what test_add_is_an_upsert_that_keeps_insertion_order exists to protect.
  • 3, the column — order-independent and durable. Its cost is the schema change, handled below.

What changed

The column is what reads project, and current() filters on it instead of on an OPTIONAL MATCH. That is both tidier and the actual fix: filtering on the edge is what returned a retracted claim as live.

The edge remains, because it is the provenance chain this backend exists for and cypher() walks it. _write reconciles it in both directions on every write — forward when this claim names a superseder that is present, backward when a stored claim names this one and could not have an edge until now. So the edge is complete once both ends have arrived, in either order. A stale edge is dropped first, because add is an upsert, for the same reason the entity edges are rebuilt rather than merged.

The migration, verified rather than assumed

CREATE NODE TABLE IF NOT EXISTS does not alter an existing table, so a database written before this has every column but superseded_by. Adding it leaves it NULL everywhere, which would read as "nothing was ever superseded" — so _migrate_superseded_by backfills from the SUPERSEDED_BY edges, which are what an older database does have, since supersede() always wrote them. (The add() path never did; that data is gone for good. This recovers what was recoverable.)

Tested against a database built with the driver in the shape the old code created, not a checked-in fixture: the link is recovered, seq does not restart and collide, and a second open is a no-op. A driver that cannot add the column raises with an explanation rather than writing rows that are quietly wrong.

Acceptance criteria

Criterion
Importing a superseded claim leaves it superseded, on all three backends, in any arrival order ✅ both orders tested
current() never returns a claim whose superseded_at is set ✅ asserted as a property over all_claims()
The three backends give the same answer, asserted by a test that compares them ✅ _shape comparison plus current()
pytest green, ruff check . clean ✅ 2,197 selected, 13 deselected

Five of the eight new tests are red without the fix; the other three guard the mechanism it adds (edge walkability, stale-edge removal on upsert, migration).

The module docstring claimed superseded_by is "an edge rather than a foreign key in a column". It is both now, and says so.

🤖 Generated with Claude Code

Closes #110, taking option 3 of the three the issue costed — persist
`superseded_by` as a column and keep the edge derived from it.

`superseded_by` was reconstructed from the SUPERSEDED_BY edge, and `add()`
cannot create an edge towards a claim the store does not have yet. In the
natural import order it does not have it: `all_claims()` returns oldest-first,
so a superseded claim arrives before the claim that superseded it. The edge was
skipped while `superseded_at` was written, so the same row said "retracted" and
`superseded_by IS NULL` said "current" — `Claim.is_current` believed the second,
and `current("svc")` returned both sides of a correction. Two of three backends
agreed; this one contradicted itself. `supersede()` was never affected because
it writes the edge itself, which is why every test that went through it passed.

The column is what reads project now, and `current()` filters on it rather than
on an OPTIONAL MATCH, which is both tidier and the actual fix — filtering on the
edge is what returned a retracted claim as live.

The edge remains, because it is the provenance chain this backend exists for and
`cypher()` walks it. `_write` reconciles it in both directions on every write:
forward when this claim names a superseder that is present, backward when a
stored claim names *this* one and could not have an edge until now. So the edge
is complete once both ends have arrived, in either order, and a stale one is
dropped first — `add` is an upsert, for the same reason the entity edges are
rebuilt rather than merged.

Option 1 (fail closed) was rejected because it breaks replaying `all_claims()`
in its own returned order, and option 2 (a stub node) because the stub reads
back as a ValidationError and has no `seq`, which is what
`test_add_is_an_upsert_that_keeps_insertion_order` is about.

The cost option 3 was costed at is the schema change, and it is handled rather
than assumed: `CREATE NODE TABLE IF NOT EXISTS` does not alter an existing
table, so `_migrate_superseded_by` adds the column and backfills it from the
edges an older database does have. Verified against a database built with the
driver in the shape the old code created, not a checked-in fixture: the link is
recovered, `seq` does not restart, and a second open is a no-op. A driver that
cannot add the column raises with an explanation instead of writing rows that
are wrong.

The module docstring said `superseded_by` is "an *edge* rather than a foreign
key in a column". It is both now, and says so.

Verified: 2197 selected, 13 deselected, ruff clean. Five of the eight new tests
are red without the fix; the other three guard the mechanism it adds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Shashankss1205
Shashankss1205 merged commit 6f79256 into main Sep 26, 2026
7 checks passed
@Shashankss1205
Shashankss1205 deleted the fix/issue-110-ladybug-superseded-by branch September 26, 2026 10:27
Shashankss1205 added a commit that referenced this pull request Sep 26, 2026
…124)

#123 merged after the release commit, so `main` carried the fix while the
0.1.8 section said nothing about it. 0.1.8 is not published and not tagged,
so the release's contents are not frozen: folding the entry in is correct and
leaves no version gap, where shipping it as 0.1.9 would have meant publishing
twice for one batch of work.

The entry carries the same weight as the other four — what broke, how it
surfaced, which options were rejected and why, and what the migration can and
cannot recover.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Shashankss1205 added a commit that referenced this pull request Sep 26, 2026
#127 and #126 merged after the release commit and neither is in the notes.
0.1.8 is still unpublished and untagged, so folding them in is correct for the
same reason #124 did it for #123.

The reclaim gets a full entry: it changes shipped behaviour and adds a public
method, and the entry is explicit that it closes a documented *limitation*
rather than a defect -- which comes to the same thing for whoever hit it. What
it refuses, and why nothing calls it automatically, are the load-bearing parts
and are stated.

The styling gate goes in the tooling paragraph, where it belongs: it changes no
shipped behaviour, it changes what a regression in shipped behaviour would be
caught by.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

memory: LadybugMemoryStore.add() drops superseded_by, so an imported retraction resurrects as current

1 participant