Skip to content

fix(storage): version raw RocksDB primary writes - #2170

Draft
kylebernhardy wants to merge 3 commits into
mainfrom
fix/recordencoder-prefixless-writes-1762
Draft

fix(storage): version raw RocksDB primary writes#2170
kylebernhardy wants to merge 3 commits into
mainfrom
fix/recordencoder-prefixless-writes-1762

Conversation

@kylebernhardy

Copy link
Copy Markdown
Member

Summary

  • enforce the RocksDB primary-store invariant at put and putSync: raw record writes receive a complete 12-byte version and zero-flags metadata prefix
  • preserve valid explicit versions, reject unusable zero/non-finite versions, and leave recordUpdater/self-versioned/non-RocksDB encoders unchanged
  • clear staged metadata across binary bypasses and failures, and prevent stale blob flags from leaking into raw rows
  • document the 0x42 classic-record collision and update the now-stale agent session guidance

Closes #1762

Verification

  • regression test fails on origin/main for missing flags metadata and missing positional version, then passes on this branch
  • 11 focused metadata/raw-write tests pass
  • full resource suite: 1,563 passing, 15 pending; an unrelated HNSW routing flake passed immediately in isolation and the full retry passed
  • build, typecheck, formatting, and changed-file lint pass
  • full integration suite passes with sequential isolation because this machine lacks the loopback alias pool
  • generic model-telemetry application on a fresh branch-built Harper root returns all six expected hdb_model_calls rows, including both formerly missing generateStream rows

Review

  • independent Claude review plus Harper-domain adjudication completed; all blocking findings addressed
  • direct Gemini CLI review passed correctness, global-state safety, data integrity, and RocksDB/LMDB boundaries after its stale-comment finding was fixed

Comment generated by kAIle (GPT-5.6)

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements versioning for raw RocksDB primary writes at the store boundary to prevent collisions between prefix-less classic records (specifically those starting with structure ID 0x42) and timestamp-prefixed records. It introduces put and putSync wrappers in PrimaryRocksDatabase that stage a monotonic version and zero-flags metadata word via stageRawPrimaryEncoding before executing the write, clearing it in a finally block. It also adds comprehensive unit tests to verify this raw-write versioning behavior. There are no review comments to address, so I have no additional feedback to provide.

this.#cache?.delete(id);
const staged = stageRawPrimaryEncoding(this.#enc, typeof options === 'number' ? options : options?.version);
try {
return super.put(id, value, options);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What: put(id, value, options) declares only 3 params and forwards exactly super.put(id, value, options). Any 4th argument a caller passes is silently dropped — it's not captured by any parameter and isn't forwarded via arguments/rest-spread.

Why it matters: resources/Table.ts:1152 calls primaryStore.put(Symbol.for('id_allocation'), {...}, Date.now(), version) — a 4-arg form where the trailing version is an ifVersion optimistic-concurrency guard (lmdb-js/rocksdb-js put(key, value, version, ifVersion)). That call site isn't wrapped in transactionSync (unlike the sibling createNewAllocation path at Table.ts:1237), so ifVersion is its only protection against a lost update when two workers race on id allocation. Before this PR, PrimaryRocksDatabase didn't override put() at all, so the call reached the native 4-arg API directly. Now it silently becomes an unconditional put, reopening the race between the getEntry read at Table.ts:1143 and this write.

Suggested fix: Accept and forward a 4th parameter (e.g. ifVersion?: any) through to super.put(id, value, options, ifVersion), mirroring whatever putSync needs for the same reason.

@claude

claude Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

One blocker found (inline): PrimaryRocksDatabase.put() only forwards 3 args to super.put(), silently dropping the ifVersion optimistic-concurrency argument used by Table.ts:1152's id-allocation write.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RecordEncoder: 0x42 prefix-less classic-record decode ambiguity keeps resurfacing — write-side guard or raw-put caller audit

1 participant