From f3ab32809fe075dc6b6842912b654a07fa5f84cd Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Thu, 20 Aug 2026 15:56:35 -0600 Subject: [PATCH 1/2] Document record-structure dictionary counts on describe_table Companion to harper core's observability change for HarperFast/harper#2220. Explains what a record structure is, why the dictionary only grows, what reaching the bound means, and how to keep the dictionary small. Co-Authored-By: Claude Opus 5 --- reference/operations-api/operations.md | 29 ++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index db0d3df0..80230bfd 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -69,6 +69,35 @@ Returns the definition of a specific table. { "operation": "describe_table", "table": "dog", "database": "dev" } ``` +Alongside the schema, the response carries the size of the table's **record-structure dictionaries** +— the physical record layouts Harper has seen for this table: + +| Field | Description | +| -------------------------- | ------------------------------------------------------------------------------------ | +| `typed_structures_enabled` | Whether random-access (typed) encoding is enabled for this table | +| `typed_structure_count` | Structures in the random-access dictionary | +| `typed_structure_limit` | Bound past which novel record shapes are stored without random-access field encoding | +| `classic_structure_count` | Structures in the classic named-record dictionary | + +A structure is minted per distinct _shape_, where shape means the ordered list of fields plus each +field's value width class — so `{a, b}` and `{b, a}` are different shapes, and so are `{v: 1}` and +`{v: 70000}`. Dictionary size therefore tracks the variety of shapes a table has ever written, not +its column count, and it only grows: structures are never pruned, because stored records, +transaction-log entries, and replication backlogs all reference them by id. + +`storage.randomAccessFields` defaults to off, so `typed_structures_enabled: false` with +`typed_structure_count: 0` is the normal state for most tables — that is typed encoding being +disabled, not spare headroom. Where it is enabled, reaching `typed_structure_limit` is not an error: +records with novel shapes past that point still write and read correctly, but are stored without +random-access field encoding, which makes reading individual fields of large records slower. Harper +logs a warning when a thread first observes the bound, but that depends on which thread served the +writes and whether it has loaded the dictionary — the counts here are the reliable signal. + +To keep the dictionary small, write records in a consistent field order, avoid making the _set_ of +present fields vary per write (nest volatile or optional fields inside one sub-object rather than +adding and removing top-level fields), and expect a small fixed number of extra shapes from numeric +fields whose values cross a width boundary. + ### `create_database` Creates a new database. From 5ec50707d471c02425083e87d9d4def33bbdd8cb Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Fri, 21 Aug 2026 15:45:31 -0600 Subject: [PATCH 2/2] Add the changed-surface version badge to the describe_table fields Repo convention (AGENTS.md): behavior changes to an existing surface carry ``. Patch-level badges have precedent in reference/ (v5.1.5, v5.1.13, v5.1.15), and main is at 5.2.4. Co-Authored-By: Claude Opus 5 --- reference/operations-api/operations.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index 80230bfd..f5209225 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -69,6 +69,8 @@ Returns the definition of a specific table. { "operation": "describe_table", "table": "dog", "database": "dev" } ``` + + Alongside the schema, the response carries the size of the table's **record-structure dictionaries** — the physical record layouts Harper has seen for this table: