S2: fix the second SSI write-skew miss (lazy-read_lsn boundary + scan on waited WRITE grants)

docs_src/api/c/txnbegin.md

@@ -97,6 +97,21 @@ The **flags** parameter must be set to 0 or by bitwise inclusively **OR**'ing to
 
   > **Fixed after 5.3.34.** Release 5.3.34 had two defects affecting serializable snapshot isolation, both now fixed: a write skew could commit when the second transaction's write landed while the first was inside <a href="txncommit.md" class="xref" title="DB_TXN-&gt;commit()">DB_TXN-&gt;commit()</a> (issue #136), and long-lived environments running many serializable transactions could exhaust the mutex region with `ENOMEM` because reader bookkeeping was not fully reclaimed (issues #137, #138). Each fix is covered by a regression test that runs in continuous integration.
 
+  **Retrying an aborted serializable transaction.** Treat `DB_SNAPSHOT_CONFLICT` and `DB_SNAPSHOT_UNSAFE` exactly as you treat `DB_LOCK_DEADLOCK`: abort the transaction and re-run the whole transaction (reads included) from the start. The second attempt sees a newer snapshot and usually succeeds; use bounded retries with a short backoff. Do not retry only the failing operation — the snapshot the transaction read from is stale, so the entire unit of work must be replayed.
+
+  > **A fixed serializability defect (cross-leaf write skew).** Through v5.3.34
+  > and until this fix, `DB_TXN_SERIALIZABLE` could *miss* a write skew between
+  > records on different leaf pages of a multi-level B-tree and let a
+  > non-serializable history commit silently — no error returned. The cause was
+  > a page-granularity bug in the SSI conflict scan (a transaction dropped its
+  > own SIREAD marker when it write-locked a page it had only read a different
+  > key on). It is now fixed and guarded by a regression test
+  > (`test/isolation/ssi_multiread_skew`), so `DB_TXN_SERIALIZABLE` prevents the
+  > skew at every B-tree depth. Serializable isolation on **Hash** remains less
+  > thoroughly qualified than on B-tree.
+
+  > **Interaction with the optimistic-read fast path (RFC 0007).** The pin-free optimistic B-tree descent (RFC 0007) is **disabled for databases opened `DB_MULTIVERSION`**, and MVCC is required for any snapshot or serializable transaction. A `DB_TXN_SNAPSHOT` or `DB_TXN_SERIALIZABLE` transaction therefore always takes the ordinary pin-and-latch descent and gains none of the optimistic-read speedup. This is by design (the optimistic path cannot observe the version chain), not a defect; it is noted so the performance trade-off of enabling MVCC is explicit.
+
 - `DB_TXN_SYNC`
 
   Synchronously flush the log when this transaction commits or prepares. This means the transaction will exhibit all of the ACID (atomicity, consistency, isolation, and durability) properties.

docs_src/guides/programmer_reference/program_errorret.md

@@ -58,11 +58,11 @@ The DB_SECONDARY_BAD error is returned if a secondary index has been corrupted.
 
 **DB_SNAPSHOT_CONFLICT**
 
-The DB_SNAPSHOT_CONFLICT error is returned to a transaction started with the <a href="../../api/c/txnbegin.md" class="olink">DB_TXN_SNAPSHOT</a> flag (serializable snapshot isolation, SSI) when Berkeley DB detects a conflicting snapshot update — an attempt to commit an update that conflicts with another concurrent snapshot transaction. The affected transaction must be aborted; the application may then retry it. This return is specific to serializable snapshot isolation.
+The DB_SNAPSHOT_CONFLICT error is returned to a transaction started with the <a href="../../api/c/txnbegin.md" class="olink">DB_TXN_SERIALIZABLE</a> flag (serializable snapshot isolation, SSI) when Berkeley DB detects a conflicting snapshot update — an attempt to commit an update that conflicts with another concurrent snapshot transaction. Handle it exactly as <a href="#program_errorret" class="olink">DB_LOCK_DEADLOCK</a>: abort the transaction and re-run the entire transaction (its reads included) from the beginning, with bounded retries and a short backoff; retrying only the failing operation is wrong because the transaction's snapshot is stale. This return is specific to serializable snapshot isolation.
 
 **DB_SNAPSHOT_UNSAFE**
 
-The DB_SNAPSHOT_UNSAFE error is returned to a transaction started with the <a href="../../api/c/txnbegin.md" class="olink">DB_TXN_SNAPSHOT</a> flag (serializable snapshot isolation, SSI) when Berkeley DB detects a potential serializability anomaly through a read/write anti-dependency: the transaction sits as the pivot of a dangerous structure (it is both the read end and the write end of anti-dependency edges among committed and running transactions) that could otherwise produce a non-serializable schedule. The affected transaction is aborted rather than allowed to commit; the application may then retry it. As with DB_SNAPSHOT_CONFLICT, this return is specific to serializable snapshot isolation.
+The DB_SNAPSHOT_UNSAFE error is returned to a transaction started with the <a href="../../api/c/txnbegin.md" class="olink">DB_TXN_SERIALIZABLE</a> flag (serializable snapshot isolation, SSI) when Berkeley DB detects a potential serializability anomaly through a read/write anti-dependency: the transaction sits as the pivot of a dangerous structure (it is both the read end and the write end of anti-dependency edges among committed and running transactions) that could otherwise produce a non-serializable schedule. The affected transaction is aborted rather than allowed to commit; as with DB_SNAPSHOT_CONFLICT, abort and re-run the whole transaction (reads included). This return is specific to serializable snapshot isolation.
 
 **DB_CONSTRAINT_VIOLATION**
 

lang/rust/libdb-sys/src/lib.rs

@@ -144,6 +144,7 @@ pub const DB_MULTIPLE_KEY: u_int32_t = 0x0000_4000;
 
 // --- txn flags --------------------------------------------------------------
 pub const DB_TXN_SNAPSHOT: u_int32_t = 0x0000_0004;
+pub const DB_TXN_SERIALIZABLE: u_int32_t = 0x0020_0000;
 pub const DB_TXN_NOWAIT: u_int32_t = 0x0000_0002;
 pub const DB_TXN_NOSYNC: u_int32_t = 0x0000_0001;
 pub const DB_TXN_WRITE_NOSYNC: u_int32_t = 0x0000_0020;
@@ -238,6 +239,7 @@ pub const CONSTANTS: &[(&str, i64)] = &[
     ("DB_MULTIPLE", DB_MULTIPLE as i64),
     ("DB_MULTIPLE_KEY", DB_MULTIPLE_KEY as i64),
     ("DB_TXN_SNAPSHOT", DB_TXN_SNAPSHOT as i64),
+    ("DB_TXN_SERIALIZABLE", DB_TXN_SERIALIZABLE as i64),
     ("DB_TXN_NOWAIT", DB_TXN_NOWAIT as i64),
     ("DB_TXN_NOSYNC", DB_TXN_NOSYNC as i64),
     ("DB_TXN_WRITE_NOSYNC", DB_TXN_WRITE_NOSYNC as i64),

lang/rust/libdb/src/lib.rs

@@ -88,6 +88,16 @@ pub enum Error {
     Deadlock,
     /// A lock was unavailable (`DB_NOWAIT` / a timeout).
     LockNotGranted,
+    /// Serializable snapshot isolation (SSI) aborted this transaction as the
+    /// pivot of a dangerous rw-antidependency structure. Like [`Deadlock`]:
+    /// abort and retry the whole transaction. [`Env::transact`] retries it.
+    /// Only occurs under `TxnOptions { serializable: true, .. }`.
+    ///
+    /// [`Deadlock`]: Error::Deadlock
+    SnapshotConflict,
+    /// Snapshot isolation could not be provided (e.g. the version a snapshot
+    /// transaction needs was already reclaimed). Retry the transaction.
+    SnapshotUnsafe,
     /// The environment panicked; it must be recovered.
     RunRecovery,
     /// A secondary index is corrupt.
@@ -107,6 +117,11 @@ impl std::fmt::Display for Error {
         match self {
             Error::Deadlock => write!(f, "deadlock (retry the transaction)"),
             Error::LockNotGranted => write!(f, "lock not granted"),
+            Error::SnapshotConflict => write!(
+                f,
+                "snapshot conflict: SSI aborted the pivot (retry the transaction)"
+            ),
+            Error::SnapshotUnsafe => write!(f, "snapshot unsafe (retry the transaction)"),
             Error::RunRecovery => write!(f, "panic: run recovery"),
             Error::SecondaryBad => write!(f, "secondary index corrupt"),
             Error::Db(c, m) | Error::Os(c, m) => write!(f, "{m} ({c})"),
@@ -137,6 +152,8 @@ fn check(code: c_int) -> Result<()> {
         0 => Ok(()),
         ffi::DB_LOCK_DEADLOCK => Err(Error::Deadlock),
         ffi::DB_LOCK_NOTGRANTED => Err(Error::LockNotGranted),
+        ffi::DB_SNAPSHOT_CONFLICT => Err(Error::SnapshotConflict),
+        ffi::DB_SNAPSHOT_UNSAFE => Err(Error::SnapshotUnsafe),
         ffi::DB_RUNRECOVERY => Err(Error::RunRecovery),
         ffi::DB_SECONDARY_BAD => Err(Error::SecondaryBad),
         c if c < 0 => Err(Error::Db(c, strerror(c))),
@@ -293,6 +310,16 @@ pub enum GetInto {
 pub struct TxnOptions {
     /// Use snapshot (MVCC) isolation for this transaction.
     pub snapshot: bool,
+    /// Use serializable snapshot isolation (SSI) for this transaction.
+    ///
+    /// This is the MVCC snapshot substrate plus Cahill rw-antidependency
+    /// detection: snapshot anomalies (write skew, the read-only-transaction
+    /// anomaly) are prevented. The pivot of a dangerous structure is aborted
+    /// at commit with [`Error::SnapshotConflict`]; retry the whole
+    /// transaction. Implies `snapshot`. Plain `snapshot` does *not* pay this
+    /// tracking cost and *does* admit SI anomalies, exactly as legacy
+    /// Berkeley DB.
+    pub serializable: bool,
     /// Do not block on a lock; fail instead of waiting.
     pub nowait: bool,
     /// Do not flush the log on commit (faster, less durable).
@@ -305,6 +332,9 @@ impl TxnOptions {
         if self.snapshot {
             f |= ffi::DB_TXN_SNAPSHOT;
         }
+        if self.serializable {
+            f |= ffi::DB_TXN_SERIALIZABLE;
+        }
         if self.nowait {
             f |= ffi::DB_TXN_NOWAIT;
         }
@@ -446,24 +476,37 @@ impl Env {
 
     /// [`Env::transact`] with explicit transaction options (e.g. `nowait` so a
     /// blocked lock fails fast as a deadlock to retry, instead of waiting for
-    /// the deadlock detector). Both [`Error::Deadlock`] and
-    /// [`Error::LockNotGranted`] are treated as retryable.
+    /// the deadlock detector). [`Error::Deadlock`], [`Error::LockNotGranted`]
+    /// and the SSI aborts [`Error::SnapshotConflict`] / [`Error::SnapshotUnsafe`]
+    /// are all treated as retryable -- so a `serializable` transaction body is
+    /// retried on a pivot abort exactly as a deadlocked one is.
     pub fn transact_opts<F, T>(&self, retries: u32, opts: TxnOptions, mut body: F) -> Result<T>
     where
         F: FnMut(&Txn<'_>) -> Result<T>,
     {
         let mut attempt = 0u32;
         loop {
             let txn = self.begin_opts(None, opts)?;
-            match body(&txn) {
-                Ok(val) => {
-                    txn.commit()?;
-                    return Ok(val);
-                }
-                Err(Error::Deadlock) | Err(Error::LockNotGranted) => {
+            // The body's result AND the commit can both raise a retryable
+            // abort (the SSI pivot check runs at commit time, so a clean body
+            // can still fail to commit with SnapshotConflict).
+            let outcome = match body(&txn) {
+                Ok(val) => match txn.commit() {
+                    Ok(()) => return Ok(val),
+                    Err(e) => Err(e), // txn already resolved by commit()
+                },
+                Err(e) => {
                     txn.abort()?;
+                    Err(e)
+                }
+            };
+            match outcome {
+                Err(Error::Deadlock)
+                | Err(Error::LockNotGranted)
+                | Err(Error::SnapshotConflict)
+                | Err(Error::SnapshotUnsafe) => {
                     if attempt >= retries {
-                        return Err(Error::Deadlock);
+                        return outcome;
                     }
                     attempt += 1;
                     // Jittered backoff so two symmetric contenders desynchronise
@@ -472,10 +515,8 @@ impl Env {
                     // real workload stays responsive.
                     backoff(attempt);
                 }
-                Err(e) => {
-                    txn.abort()?;
-                    return Err(e);
-                }
+                Err(e) => return Err(e),
+                Ok(_) => unreachable!(),
             }
         }
     }
@@ -686,6 +727,8 @@ pub mod flag {
     pub const TXN_NOSYNC: u32 = ffi::DB_TXN_NOSYNC;
     /// Default new transactions to snapshot (MVCC) isolation.
     pub const TXN_SNAPSHOT: u32 = ffi::DB_TXN_SNAPSHOT;
+    /// Default new transactions to serializable snapshot isolation (SSI).
+    pub const TXN_SERIALIZABLE: u32 = ffi::DB_TXN_SERIALIZABLE;
 }
 
 /// Cursor positioning opcodes, for [`Cursor::pget`].

lang/zig/src/consts.zig

@@ -42,6 +42,7 @@ pub const DB_RENUMBER: u32 = 0x00000080;
 pub const DB_TXN_NOSYNC: u32 = 0x00000001;
 pub const DB_TXN_NOWAIT: u32 = 0x00000002;
 pub const DB_TXN_SNAPSHOT: u32 = 0x00000004;
+pub const DB_TXN_SERIALIZABLE: u32 = 0x00200000;
 pub const DB_TXN_SYNC: u32 = 0x00000008;
 pub const DB_TXN_WRITE_NOSYNC: u32 = 0x00000020;
 pub const DB_READ_UNCOMMITTED: u32 = 0x00000200;
@@ -106,6 +107,8 @@ pub const DB_SEQ_WRAP: u32 = 0x00000008;
 pub const DB_NOTFOUND: c_int = -30988;
 pub const DB_KEYEXIST: c_int = -30994;
 pub const DB_LOCK_DEADLOCK: c_int = -30993;
+pub const DB_SNAPSHOT_CONFLICT: c_int = -30968;
+pub const DB_SNAPSHOT_UNSAFE: c_int = -30967;
 pub const DB_BUFFER_SMALL: c_int = -30999;
 pub const DB_RUNRECOVERY: c_int = -30973;
 pub const DB_SECONDARY_BAD: c_int = -30972;
@@ -141,6 +144,7 @@ pub const all = [_]Entry{
     .{ .name = "DB_TXN_NOSYNC", .value = DB_TXN_NOSYNC },
     .{ .name = "DB_TXN_NOWAIT", .value = DB_TXN_NOWAIT },
     .{ .name = "DB_TXN_SNAPSHOT", .value = DB_TXN_SNAPSHOT },
+    .{ .name = "DB_TXN_SERIALIZABLE", .value = DB_TXN_SERIALIZABLE },
     .{ .name = "DB_TXN_SYNC", .value = DB_TXN_SYNC },
     .{ .name = "DB_TXN_WRITE_NOSYNC", .value = DB_TXN_WRITE_NOSYNC },
     .{ .name = "DB_READ_UNCOMMITTED", .value = DB_READ_UNCOMMITTED },
@@ -185,6 +189,8 @@ pub const all = [_]Entry{
     .{ .name = "DB_NOTFOUND", .value = DB_NOTFOUND },
     .{ .name = "DB_KEYEXIST", .value = DB_KEYEXIST },
     .{ .name = "DB_LOCK_DEADLOCK", .value = DB_LOCK_DEADLOCK },
+    .{ .name = "DB_SNAPSHOT_CONFLICT", .value = DB_SNAPSHOT_CONFLICT },
+    .{ .name = "DB_SNAPSHOT_UNSAFE", .value = DB_SNAPSHOT_UNSAFE },
     .{ .name = "DB_BUFFER_SMALL", .value = DB_BUFFER_SMALL },
     .{ .name = "DB_RUNRECOVERY", .value = DB_RUNRECOVERY },
     .{ .name = "DB_SECONDARY_BAD", .value = DB_SECONDARY_BAD },

lang/zig/src/libdb.zig

@@ -42,6 +42,13 @@ const safe = switch (builtin.mode) {
 pub const Error = error{
     /// `DB_LOCK_DEADLOCK` -- abort the txn and retry (see `Env.transact`).
     Deadlock,
+    /// `DB_SNAPSHOT_CONFLICT` -- serializable snapshot isolation (SSI) aborted
+    /// this transaction as the pivot of a dangerous structure. Abort and retry
+    /// the whole transaction, exactly like `Deadlock`. `Env.transact` retries
+    /// it. Only occurs under `DB_TXN_SERIALIZABLE`.
+    SnapshotConflict,
+    /// `DB_SNAPSHOT_UNSAFE` -- snapshot isolation could not be provided; retry.
+    SnapshotUnsafe,
     /// `DB_RUNRECOVERY` -- the environment panicked; it must run recovery.
     RunRecovery,
     /// `DB_NOTFOUND` where a value was required.
@@ -69,6 +76,8 @@ fn check(code: c_int) Error!void {
     if (code == 0) return;
     return switch (code) {
         consts.DB_LOCK_DEADLOCK => Error.Deadlock,
+        consts.DB_SNAPSHOT_CONFLICT => Error.SnapshotConflict,
+        consts.DB_SNAPSHOT_UNSAFE => Error.SnapshotUnsafe,
         consts.DB_RUNRECOVERY => Error.RunRecovery,
         consts.DB_NOTFOUND => Error.NotFound,
         consts.DB_KEYEXIST => Error.KeyExist,
@@ -78,6 +87,14 @@ fn check(code: c_int) Error!void {
     };
 }
 
+/// Whether an error is a transient abort that `Env.transact` should retry:
+/// a deadlock or either SSI snapshot abort. All three mean "abort the whole
+/// transaction and run it again".
+fn isRetryable(e: Error) bool {
+    return e == Error.Deadlock or e == Error.SnapshotConflict or
+        e == Error.SnapshotUnsafe;
+}
+
 /// The library version at run time.
 pub fn version() struct { major: i32, minor: i32, patch: i32 } {
     var a: c_int = 0;
@@ -285,11 +302,18 @@ pub const Env = struct {
         while (true) : (attempt += 1) {
             var txn = try self.begin(null, consts.DB_TXN_NOWAIT);
             if (op(ctx, &txn)) |_| {
-                try txn.commit();
-                return;
+                // The commit can itself raise a retryable abort: under SSI
+                // (DB_TXN_SERIALIZABLE, e.g. an env default) the pivot check
+                // runs at commit time, so a clean `op` can still fail here.
+                if (txn.commit()) |_| {
+                    return;
+                } else |ce| {
+                    if (isRetryable(ce) and attempt < max_retries) continue;
+                    return ce;
+                }
             } else |e| {
                 txn.abort();
-                if (e == Error.Deadlock and attempt < max_retries) continue;
+                if (isRetryable(e) and attempt < max_retries) continue;
                 return e;
             }
         }