Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs_src/_data/releases.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ historical = ["v5.3.28", "v4.6.21-SSI"]

[notes]
"v2026.10.8" = "Regular release: <strong>faster transactional writes</strong> &mdash; the deadlock detector skips its graph when no deadlock is possible, and inserts stop locking idle cursor queues (btree inserts 1.43&times; at 4 threads, recno read-modify-write 1.88&times;); typed data and constraints (<code>--enable-typed</code>), Rust and Zig bindings. Formats, soname and ABI unchanged"
"v2026.10.7.2" = "<strong>LTS patch release</strong> of the 2026.10.7 line, fixes only, formats unchanged: <code>DB_TXN_SERIALIZABLE</code> could <strong>commit a write skew</strong> (three missed conflicts, now caught), recovery of many prepared transactions ran <strong>out of lockers</strong> with lockers free on other stripes, and an equal-key put over a deleted key or an all-deleted duplicate set <strong>kept the deleted key&#39;s bytes</strong> under a non-identity <code>bt_compare</code>"
"v2026.10.7.1" = "<strong>LTS patch release</strong> of the 2026.10.7 line, fixes only, formats unchanged: a reverse cursor walk (<code>DB_PREV</code>) could <strong>silently skip live records</strong> next to deleted ones (a strict-aliasing miscompile, seen with clang on aarch64), <code>DB_TRUNCATE</code> with a subdatabase name <strong>destroyed the whole file</strong> (now refused with <code>EINVAL</code>), and a lock-order inversion when discarding a written file from the cache, which a debug build reported as a panic"
"v2026.10.7" = "<strong>First LTS release</strong>: the 2026.10.x line freezes the region, database and log formats, the soname and the ABI. On macOS arm64 the hybrid mutex is selected again (region format changes there only; see the 2026.10.7 upgrade note); every other platform is byte-identical to 2026.10.6"
"v2026.10.6" = "<strong>BREAKING: the environment region format changes</strong> (recover, remove the regions, restart). RFC 0015 scalability work and the AArch64 hybrid mutex, which fixes the FreeBSD/Linux arm64 multi-process limitation (B3)"
Expand Down
1 change: 1 addition & 0 deletions docs_src/guides/upgrading/_meta.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ order = [
"introduction",
"upgrade_process",
"upgrade_2026_10_8",
"upgrade_2026_10_7_2",
"upgrade_2026_10_7_1",
"upgrade_2026_10_7",
"upgrade_2026_10_6",
Expand Down
42 changes: 42 additions & 0 deletions docs_src/guides/upgrading/upgrade_2026_10_7_2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
---
title: "Upgrading to libdb 2026.10.7.2"
api-name: "Upgrading to libdb 2026.10.7.2"
---
## Upgrading to libdb 2026.10.7.2

libdb 2026.10.7.2 is the second patch release of the 2026.10.7 long-term support (LTS) line. It contains bug fixes only.

**No upgrade procedure is needed.** The shared-memory region format, the database and log formats, the soname and the public ABI are the same as 2026.10.7 and 2026.10.7.1 on every platform. An environment created by either opens with 2026.10.7.2 and the other way round, and processes from all three releases can share one environment. Install it in place of 2026.10.7.1.

### Build signatures

The signature depends on the platform and the configure options. On each
platform below, 2026.10.7, 2026.10.7.1 and 2026.10.7.2 built the same way print
the same value.

| Platform (default `configure`) | 2026.10.7.1 | 2026.10.7.2 |
| --- | --- | --- |
| Linux x86_64 (Debian 12, gcc) | `0x841f4ea8` | `0x841f4ea8` |
| FreeBSD aarch64 (14.5, clang) | `0x56a06689` | `0x56a06689` |
| FreeBSD x86_64, macOS arm64, Windows | unchanged | unchanged |

### What it fixes

**`DB_TXN_SERIALIZABLE` committed write skews.** Serializable snapshot isolation (SSI) is meant to abort one of two transactions that each read what the other writes. It missed three cases, so both transactions committed a history that no serial order produces:

- On a B-tree with more than one level, a transaction that write-locked a page dropped its own read marker on that page, even when it had read a different record there. A later writer of that page then saw no reader.
- A writer that had to wait behind another writer on the same page, or that re-acquired or upgraded a lock it already held, never checked the page's read markers.
- A reader that read a page after a writer had already locked it and checked the page was never seen by that writer, because a read marker never blocks and so never prompts a second check.

Each is fixed: a transaction keeps its own markers, every way a write lock is granted checks the markers, and a reader now records the conflict with the writers already holding the page. Measured with the Adya cycle checker (`test/isolation/test_ssi_cycle`) at several records per leaf, 40 seeds each, 2026.10.7.1 committed a non-serializable history in 4 of 40 runs at 2 threads, 23 of 40 at 16 threads (B-tree) and 16 of 40 at 16 threads (Hash). 2026.10.7.2 commits none, at the same throughput. If you use `DB_TXN_SERIALIZABLE`, upgrade.

**Recovery of many prepared transactions ran out of lockers.** Recovery creates one locker per prepared transaction. Since 2026.10.5 each of the 64 locker stripes has its own free list, and a stripe that ran dry could not borrow an idle locker from another stripe once the total reached `lk_max_lockers`. Recovery then failed with `BDB2055 Lock table is out of available locker entries` and panicked, even with lockers free elsewhere. A dry stripe now takes a free locker from another stripe before it gives up. The TCL test `recd015` (10,000 prepared transactions) failed on 2026.10.5 through 2026.10.7.1 and passes again.

**An equal-key put over a deleted key kept the deleted key's bytes.** In a B-tree whose `bt_compare` treats different byte strings as equal (for example a case-insensitive comparison, `"abc"` equal to `"ABC"`), a put of `"ABC"` over a deleted `"abc"` that a cursor still referenced brought the pair back under the old bytes, `"abc"`. The stored key depended on whether an unrelated cursor happened to be open. The same happened to a duplicate set whose members were all deleted while a cursor referenced one of them. In SQL this showed up as a `COLLATE NOCASE` index or a WITHOUT ROWID primary key keeping the old spelling after an `UPDATE`. The put now stores the new key's bytes. For a duplicate set the set's one shared key is rewritten in place, so every member and every cursor sees the new spelling, and `DB->verify` passes. One case is unchanged: a duplicate set whose shared key is long enough to be stored on an overflow page keeps the old bytes. The set stays valid; see `test/KNOWN-ISSUES.md`.

Each fix has a regression test that fails on 2026.10.7.1 and passes on 2026.10.7.2: `test/isolation/ssi_multiread_skew.c`, `test/isolation/test_ssi_cycle.c` and `test/isolation/ssi_livelock.c` (the isolation tier), `test/db/run_lock_stripe_exhaustion.sh`, `test/db/run_r1_revive.sh`, `test/db/run_r2_dup_revive.sh` and `test/db/run_r2_fuzz.sh`.

### Also in this release

- `run_qam_extent_vrfy` allows 600 seconds instead of 180. GitHub's Ubuntu 26.04 runner image needs about 181 seconds for its 775 extent-file syncs.
- A recno duplicate tree is descended with the recno internal-page macro, not the btree one.
50 changes: 50 additions & 0 deletions test/audit/QUAL-2026.10.7.2-freebsd-aarch64.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# v2026.10.7.2 qualification: freebsd-aarch64 (FreeBSD 14.5-STABLE arm64)
base: defea6df Merge pull request #221 from berkeleydb/backport/2026.10.7/fixes-1
cand: 1943cc9b release: libdb 2026.10.7.2 (LTS patch)

## L1 region signature (default configure)
base: 0x56a06689
cand: 0x56a06689
L1 PASS

## L2 soname, exported symbols, .text, handle sizes
base soname: libdb-2026.0.so
base exported symbols: 1848
base .text: 1061308
cand soname: libdb-2026.0.so
cand exported symbols: 1848
cand .text: 1064460
symbols removed: 0 added: 0
handle_sizes (cand): run_handle_sizes.sh: PASS

## L3 fixed bugs: base / release (each test from the candidate tree, run against both libraries)
run_r1_revive: FAIL / PASS
run_r2_dup_revive: FAIL / PASS
run_r2_fuzz: PASS / PASS
run_lock_stripe_exhaustion: FAIL / PASS
isolation tier (incl. ssi_cycle_btree, ssi_multiread_skew, ssi_livelock): base rc=1 / release rc=0
base failing cells:
cand failing cells:

## gate 1 test/db tier (release, diagnostic)
cand rel: rc=1 pass=35 skip=4 fail=1
run_qam_extent_vrfy.sh: FAIL (rc=124)
cand diag: rc=1 pass=35 skip=4 fail=1
run_qam_extent_vrfy.sh: FAIL (rc=124)

## gate 8 soak (SSI changes SIREAD marker lifetime)
soak rc=0: 5 workload(s) run, 0 unexpected outcome(s)

## done 2026-11-04T00:16:30Z

## gate 1 rerun, run_qam_extent_vrfy TIMEOUT=1200 (see QUAL-2026.10.7.2.md)
cand rel (qam TIMEOUT=1200): rc=0 pass=37 skip=4 fail=0
run_u9_serializable.sh: SKIP no javac on this host
run_u8_backup_config.sh: SKIP no javac on this host
run_mvcc_mtx_doublefree.sh: SKIP failchk reclaimed no mutexes (0 BDB2017 lines), so the F6 scenario was never constructed -- this run proves nothing either way
run_meson_autoconf_parity.sh: SKIP no meson on this host
cand diag (qam TIMEOUT=1200): rc=0 pass=37 skip=4 fail=0
run_u9_serializable.sh: SKIP no javac on this host
run_u8_backup_config.sh: SKIP no javac on this host
run_mvcc_mtx_doublefree.sh: SKIP failchk reclaimed no mutexes (0 BDB2017 lines), so the F6 scenario was never constructed -- this run proves nothing either way
run_meson_autoconf_parity.sh: SKIP no meson on this host
49 changes: 49 additions & 0 deletions test/audit/QUAL-2026.10.7.2-freebsd-x86_64.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# v2026.10.7.2 qualification: freebsd-x86_64 (FreeBSD 14.5-STABLE amd64)
base: defea6df Merge pull request #221 from berkeleydb/backport/2026.10.7/fixes-1
cand: 1943cc9b release: libdb 2026.10.7.2 (LTS patch)

## L1 region signature (default configure)
base: 0x56a06689
cand: 0x56a06689
L1 PASS

## L2 soname, exported symbols, .text, handle sizes
base soname: libdb-2026.0.so
base exported symbols: 1848
base .text: 1156684
cand soname: libdb-2026.0.so
cand exported symbols: 1848
cand .text: 1160828
symbols removed: 0 added: 0
handle_sizes (cand): run_handle_sizes.sh: PASS

## L3 fixed bugs: base / release (each test from the candidate tree, run against both libraries)
run_r1_revive: FAIL / PASS
run_r2_dup_revive: FAIL / PASS
run_r2_fuzz: PASS / PASS
run_lock_stripe_exhaustion: FAIL / PASS
isolation tier (incl. ssi_cycle_btree, ssi_multiread_skew, ssi_livelock): base rc=1 / release rc=0
base failing cells:
cand failing cells:

## gate 1 test/db tier (release, diagnostic)
cand rel: rc=1 pass=35 skip=4 fail=1
run_qam_extent_vrfy.sh: FAIL (rc=124)
cand diag: rc=1 pass=35 skip=4 fail=0

## gate 8 soak (SSI changes SIREAD marker lifetime)
soak rc=0: 5 workload(s) run, 0 unexpected outcome(s)

## done 2026-11-04T00:10:25Z

## gate 1 rerun, run_qam_extent_vrfy TIMEOUT=1200 (see QUAL-2026.10.7.2.md)
cand rel (qam TIMEOUT=1200): rc=0 pass=37 skip=4 fail=0
run_u9_serializable.sh: SKIP no javac on this host
run_u8_backup_config.sh: SKIP no javac on this host
run_mvcc_mtx_doublefree.sh: SKIP failchk reclaimed no mutexes (0 BDB2017 lines), so the F6 scenario was never constructed -- this run proves nothing either way
run_meson_autoconf_parity.sh: SKIP no meson on this host
cand diag (qam TIMEOUT=1200): rc=0 pass=37 skip=4 fail=0
run_u9_serializable.sh: SKIP no javac on this host
run_u8_backup_config.sh: SKIP no javac on this host
run_mvcc_mtx_doublefree.sh: SKIP failchk reclaimed no mutexes (0 BDB2017 lines), so the F6 scenario was never constructed -- this run proves nothing either way
run_meson_autoconf_parity.sh: SKIP no meson on this host
36 changes: 36 additions & 0 deletions test/audit/QUAL-2026.10.7.2-linux-x86_64.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# v2026.10.7.2 qualification: linux-x86_64 (Linux 6.1.0-54-cloud-amd64 x86_64)
base: defea6df Merge pull request #221 from berkeleydb/backport/2026.10.7/fixes-1
cand: 1943cc9b release: libdb 2026.10.7.2 (LTS patch)

## L1 region signature (default configure)
base: 0x841f4ea8
cand: 0x841f4ea8
L1 PASS

## L2 soname, exported symbols, .text, handle sizes
base soname: libdb-2026.0.so
base exported symbols: 1846
base .text: 1302044
cand soname: libdb-2026.0.so
cand exported symbols: 1846
cand .text: 1306636
symbols removed: 0 added: 0
handle_sizes (cand): run_handle_sizes.sh: PASS

## L3 fixed bugs: base / release (each test from the candidate tree, run against both libraries)
run_r1_revive: FAIL / PASS
run_r2_dup_revive: FAIL / PASS
run_r2_fuzz: PASS / PASS
run_lock_stripe_exhaustion: FAIL / PASS
isolation tier (incl. ssi_cycle_btree, ssi_multiread_skew, ssi_livelock): base rc=1 / release rc=0
base failing cells:
cand failing cells:

## gate 1 test/db tier (release, diagnostic)
cand rel: rc=0 pass=38 skip=3 fail=0
cand diag: rc=0 pass=38 skip=3 fail=0

## gate 8 soak (SSI changes SIREAD marker lifetime)
soak rc=0: 5 workload(s) run, 0 unexpected outcome(s)

## done 2026-11-03T23:55:33Z
114 changes: 114 additions & 0 deletions test/audit/QUAL-2026.10.7.2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# v2026.10.7.2 qualification (LTS patch release, 2026-11-03)

The second patch release of the `2026.10.7` LTS line, qualified against the
gates for an LTS patch release in [`QUALIFICATION-MATRIX.md`](QUALIFICATION-MATRIX.md)
(gates 1–5 and 7, plus L1–L4). Gate 8, the soak, also ran, because the SSI fix
changes how long SIREAD markers live. The candidate was compared with the base
tag `v2026.10.7.1` (`defea6df0`), both built the same way on the same host.

**Candidate:** `release/2026.10.7.2`, head `1943cc9b3` (PR #260).

**Hosts:** EC2, on-demand:

| Host | OS | vCPU | Compiler |
|---|---|---|---|
| Linux x86_64 | Debian 12, c6i.8xlarge | 32 | gcc |
| FreeBSD x86_64 | FreeBSD 14.5, c6i.4xlarge | 16 | clang |
| FreeBSD aarch64 | FreeBSD 14.5, c7g.4xlarge | 16 | clang |

The raw per-host summaries are kept alongside this file:

- `QUAL-2026.10.7.2-linux-x86_64.txt`
- `QUAL-2026.10.7.2-freebsd-x86_64.txt`
- `QUAL-2026.10.7.2-freebsd-aarch64.txt`

## Result: GO

| Gate | Linux x86_64 | FreeBSD x86_64 | FreeBSD aarch64 |
|---|---|---|---|
| **L1** region signature, base = release | `0x841f4ea8` = `0x841f4ea8` | `0x56a06689` = `0x56a06689` | `0x56a06689` = `0x56a06689` |
| **L2** handle sizes | PASS | PASS | PASS |
| **L2** soname, exported symbols | `libdb-2026.0.so`; 1846 → 1846, 0 removed, 0 added | `libdb-2026.0.so`; 1848 → 1848, 0 / 0 | `libdb-2026.0.so`; 1848 → 1848, 0 / 0 |
| **L2** `.text` (base → release) | 1,302,044 → 1,306,636 | 1,156,684 → 1,160,828 | 1,061,308 → 1,064,460 |
| **L3** `r1_revive`: base / release | **FAIL / PASS** | **FAIL / PASS** | **FAIL / PASS** |
| **L3** `r2_dup_revive`: base / release | **FAIL / PASS** | **FAIL / PASS** | **FAIL / PASS** |
| **L3** `r2_fuzz`: base / release | PASS / PASS | PASS / PASS | PASS / PASS |
| **L3** `lock_stripe_exhaustion`: base / release | **FAIL / PASS** | **FAIL / PASS** | **FAIL / PASS** |
| **L3** isolation tier: base / release | **rc=1 / rc=0** | **rc=1 / rc=0** | **rc=1 / rc=0** |
| **L3** `test_ssi_cycle` btree 2 threads, non-serializable / 40: base / release | **7 / 0** | **5 / 0** | **9 / 0** |
| **L3** `test_ssi_cycle` btree 16 threads | **21 / 0** | **20 / 0** | **27 / 0** |
| **L3** `test_ssi_cycle` hash 16 threads | **21 / 0** | **22 / 0** | **16 / 0** |
| `ssi_livelock`: SSI commits as % of SI, base / release | 99% / 98% | 73–102% / 92% | 90% / 91% |
| **1** `test/db` tier, release / release diagnostic | 38 pass, 3 skip, 0 fail / 38, 3, 0 | 37 pass, 4 skip, 0 fail / 37, 4, 0 † | 37 pass, 4 skip, 0 fail / 37, 4, 0 † |
| **8** soak | 5 workloads, 0 unexpected | 5, 0 | 5, 0 |
| **2** `s_validate` (22 checkers), `s_execbits`, `s_chk_copyright`, `s_chk_licence`, `s_message_id` | rc=0 | (bash-only; Linux covers) | (bash-only; Linux covers) |
| **4** CI region-signature and ABI gates (PR #260) | success | | |
| **5** CI at job level (PR #260, head `1943cc9b3`) | 64 success, 5 skipped, **0 failed** | | |
| **L4** every LTS commit traces to `master` | 17 cherry-picks, all with a `-x` trailer naming a `master` commit; 2 LTS-only (MANIFEST, release) | | |

The skips are environmental: no JDK (`u8_backup_config`, `u9_serializable`),
no meson on FreeBSD (`meson_autoconf_parity`), and `mvcc_mtx_doublefree` when
failchk reclaims no mutex, so its scenario is never built.

**† `run_qam_extent_vrfy` on FreeBSD.** The first tier run on both FreeBSD hosts
timed it out (`rc=124` at its 600 s limit). It is I/O-bound by design: about 775
syncs, one per extent file. Run alone on the same hosts it takes 602–640 s, and
the **base tag takes the same time** (x86_64: base 636 s, release 640 s), so the
release did not slow it down. These hosts use their EBS root (UFS with soft
updates), not the instance store the 2026.10.7.1 hosts used. The tier was rerun
with `TIMEOUT=1200` for that one runner, and those are the counts above. The
limit on the line stays at 600 s; GitHub's runners need about 181 s.

**L3, the SSI fix.** The isolation tier fails on the base on every host with
`ssi_multiread_skew` committing the cross-leaf write skew in 40/40 attempts, and
passes on the release. The cycle checker (`CYC_VALPAD=200`, several records per
leaf, 40 seeds, 2 s each) found non-serializable histories on the base on every
host and configuration, and none on the release. No release seed tripped the
throughput floor or the watchdog.

**`ssi_livelock` on FreeBSD prints `BDB3010`.** On both FreeBSD hosts the
release run printed `BDB3010 error getting a page for writing` 30–89 times in
3 s. A probe that counts return codes showed that every transaction that printed
it returned `DB_SNAPSHOT_CONFLICT` or `DB_SNAPSHOT_UNSAFE` and aborted normally,
and none returned another code. `__memp_dirty` suppresses that message for a
deadlock but not for an SSI abort, and the SSI fix raises the abort rate under
contention. The message is noise, not a failure. The fix is #261, for the next
patch release.

**Gate 6** (the metal benchmark) was not run. The SSI change adds a scan of the
page's write holders when a SIREAD marker is granted, which only applies to
`DB_TXN_SERIALIZABLE` transactions. `ssi_livelock` and the cycle checker show
the same commit throughput as the base. The other fixes are on rare paths: a put
over a deleted key, and locker allocation when a stripe runs dry.

## L4: every commit since v2026.10.7.1

17 commits are cherry-picks with `-x` trailers, and 2 are LTS-only:

- `84717f412`, LTS-only: this line's `test/MANIFEST`. The cherry-pick of
`b803b8cd1` took master's copy, which listed eight runners only master has.
- `1943cc9b3`, LTS-only: the release commit.

All 17 `-x` trailers resolve to commits on `master`
(`git merge-base --is-ancestor <trailer> origin/master`, checked after the
master PRs merged):

| master PR | commits |
|---|---|
| #242 (CI 26.04) | 81e44c28c |
| R1 | 50ef9c73f |
| #244 (R2) | 76169312b, 9d6d00599, b9064d512, ca66e9f2a, 2e6ae3f7e, 07a240a25, 2d24d212a, 17116871e, d816880e0, ab11f6ff2 |
| #255 (recd015 lockers) | 6ff1561b1 |
| #257 (SSI) | cced5c481, 697c965eb, 2097393af, 12ed935bc |

## What the release fixes

| PR (LTS / master) | Fix | Who sees it |
|---|---|---|
| #258 / #257 | `DB_TXN_SERIALIZABLE` committed write skews (three missed conflicts) | every SSI user |
| #256 / #255 | recovery of many prepared transactions ran out of lockers with lockers free on other stripes | 2026.10.5 and later |
| #239 / R1 | an equal-key put over a deleted key kept the deleted key's bytes under a non-identity `bt_compare` | custom comparators, e.g. `COLLATE NOCASE` in SQL |
| #245 / #244 | the same, for a duplicate set whose members were all deleted | custom comparators with `DB_DUP`/`DB_DUPSORT` |

Also backported: #248 (the `run_qam_extent_vrfy` timeout for the Ubuntu 26.04
runner).
5 changes: 3 additions & 2 deletions test/audit/QUALIFICATION-MATRIX.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,9 @@ Gate 6 (the cross-release metal benchmark) is skipped unless a fix touches a
hot path. Gate 8 (the soak) is skipped unless a fix touches resource lifetime.
The release notes say which gates ran.

The first LTS patch release, `v2026.10.7.1`, is recorded in
[`QUAL-2026.10.7.1.md`](QUAL-2026.10.7.1.md).
The LTS patch releases are recorded in
[`QUAL-2026.10.7.1.md`](QUAL-2026.10.7.1.md) and
[`QUAL-2026.10.7.2.md`](QUAL-2026.10.7.2.md).

## Additional gates for the LTS declaration

Expand Down
Loading