Skip to content

Document record-structure dictionary counts on describe_table - #633

Draft
kriszyp wants to merge 1 commit into
mainfrom
kris/2220-structure-metric
Draft

Document record-structure dictionary counts on describe_table#633
kriszyp wants to merge 1 commit into
mainfrom
kris/2220-structure-metric

Conversation

@kriszyp

@kriszyp kriszyp commented Aug 20, 2026

Copy link
Copy Markdown
Member

Companion to the core change for HarperFast/harper#2220 — Per-table msgpackr structure dictionaries grow unbounded and unobservably, which adds four fields to describe_table: typed_structures_enabled, typed_structure_count, typed_structure_limit, classic_structure_count.

Documents what a record structure is, why the dictionary only ever grows, what reaching the bound means, and how an application keeps it small.

For the human reviewer

  • The growth model is the part worth checking. A structure is minted per distinct shape = ordered field list × each field's value width class — so {a, b} and {b, a} are different shapes, and so are {v: 1} and {v: 70000}. This is measured against the installed deps, not inferred: 4 key orders → 4 structures; one two-field shape across 11 value kinds → 5 structures. The issue that prompted this attributed growth to realized field subsets alone, which understates it.
  • typed_structures_enabled needs the surrounding sentence. storage.randomAccessFields defaults off, so most tables show typed_structure_count: 0 against a limit of 256. Without the explanation an operator reads that as spare headroom rather than the feature being disabled, so the paragraph saying so is load-bearing.
  • The warning's scope is stated honestly rather than simply. It fires per encoder from the post-save path, so several workers can each warn for one table and a worker that never saves after loading a full dictionary stays silent. The text says the counts are the reliable signal and the log is the heads-up, rather than implying one warning per table.
  • No analytics metric is documented, deliberately. The core change originally folded these counts onto the table-size metric; that metric turns out to have no RocksDB emitter at all (The table-size analytics metric is never emitted on RocksDB harper#2249), so documenting it here would point operators at a series that never appears.

Verification

Prettier clean. Claims cross-checked against the core diff and against live measurements from the core PR's integration suite (a saturated table reports typed 256/256, classic 32).

Companion to harper core's observability change for HarperFast/harper#2220.
Explains what a record structure is, why the dictionary only grows, what
reaching the bound means, and how to keep the dictionary small.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the 'describe_table' API documentation to detail the new record-structure dictionary fields in the response. The reviewer suggested adding a version badge to indicate this behavior change, as per the repository's documentation guidelines.

{ "operation": "describe_table", "table": "dog", "database": "dev" }
```

Alongside the schema, the response carries the size of the table's **record-structure dictionaries**

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

According to the repository's general rules, when documenting behavior changes to an existing surface (such as adding new fields to the describe_table response), we should use the <VersionBadge type="changed" version="vX.Y.0" /> format. Please add the appropriate version badge to indicate this change.

Suggested change
Alongside the schema, the response carries the size of the table's **record-structure dictionaries**
Alongside the schema, the response carries the size of the table's **record-structure dictionaries** <VersionBadge type="changed" version="v5.3.0" />
References
  1. Use the <VersionBadge type="changed" version="vX.Y.0" /> format when documenting behavior changes to existing surface, as prescribed in the repository's guidelines.

@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-633

This preview will update automatically when you push new commits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant