Skip to content

perf: decode large documents with a fraction of the memory, with a benchmark - #18

Merged
exhuma merged 21 commits into
mainfrom
perf/large-document-decoding
Oct 26, 2026
Merged

exhuma merged 21 commits into
mainfrom
perf/large-document-decoding

Conversation

@exhuma

@exhuma exhuma commented Oct 26, 2026 •

Copy link
Copy Markdown
Owner

Makes large documents cheap to decode and adds a benchmark against pyasn1, asn1tools, asn1crypto and pycrate.

Stacked on #15 (it extends the "Choosing x690" page from that PR). Merge #15 first, then retarget this PR to main.

Changes

  • Indexing a decoded Sequence is quadratic #16 – sequence indexing: a decoded Sequence records the start of each item once, in a compact array, and decodes items on demand. Indexing was quadratic: reading 200 items by index took 40,200 header reads, now at most 600. Iteration no longer builds the full item list first, and decode_raw no longer copies the value to test for emptiness.
  • Declare __slots__ on all types to reduce memory per value #17 – __slots__: every type declares __slots__, and bounds is a slot. A decoded Integer takes 120 bytes instead of 472. Custom types without slots keep working.
  • Benchmark (benchmarks/): four workloads (stream, full, partial, indexed) on synthetic CDR-like data, one process per case, memory measured on top of the input. uv run --group bench python benchmarks/run_all.py writes a JSON report with machine details; benchmarks/report.py renders and compares reports.
  • Reports: benchmarks/results/: before and after on a small machine (2 cores, 3.7 GB, 2 GB limit per case) and on a desktop (12 threads, 31 GB, no limit).
  • Docs: performance section on the "Choosing x690" page, led by the desktop run; changelog entries.

Results at 1,000,000 records (80 MB)

Desktop (Ryzen 5 5600X, 31 GB, CPython 3.12, no limit; user-desktop-*.json):

x690 workload Before After
full document 14.4 s / 232 MB 10.0 s / 8 MB
last record only 2.2 s / 233 MB 0.71 s / 8 MB
every record by index 167 s for 10,000 records 10.5 s
stream 13.8 s 9.3 s

For a whole document, x690 is now the fastest of the five: asn1crypto 12.9 s / 1,957 MB, asn1tools 13.2 s / 1,436 MB, pycrate 70 s / 5,494 MB, pyasn1 146 s / 9,569 MB. On streams asn1crypto stays fastest (6.3 s).

Small machine (2 cores, 3.7 GB, CPython 3.13, 2 GB limit; constrained-*.json):

x690 workload Before After
full document 32 s / 558 MB 22 s / 8 MB
last record only 7.2 s / 557 MB 1.6 s / 8 MB
every record by index 409 s for 10,000 records 22 s
stream 31 s 21 s

pycrate and pyasn1 run out of memory on whole documents there. The other libraries' numbers stayed within a few percent between before and after on both machines.

Verification

  • pytest: 459 passed, coverage 99.69 %. New tests for indexing, slicing, offsets inside a larger buffer, a linear bound on header reads (fails on the old code with 40,200 reads), and the absence of instance dictionaries.
  • mypy --strict, pre-commit run --all-files, sphinx-build -W --keep-going: clean.

Reproduce

$ git checkout ed64712   # benchmark without the fixes
$ uv run --group bench python benchmarks/run_all.py --label <name>-before
$ git checkout perf/large-document-decoding
$ uv run --group bench python benchmarks/run_all.py --label <name>-after
$ uv run python benchmarks/report.py benchmarks/results/<name>-before-*.json benchmarks/results/<name>-after-*.json

Closes #16
Closes #17

🤖 Generated with Claude Code

exhuma and others added 16 commits October 26, 2026 12:28
Replace alabaster with the michel-albert-sphinx extension (v0.3.1),
which runs on sphinx-immaterial, and drop the unused sphinx-rtd-theme.
Build the docs with --keep-going so every warning shows up in one run.

sphinx-immaterial links TypeVars in signatures to type parameters; the
module-level TypeVars in x690.types have no such target, so that
warning category is suppressed.

Co-Authored-By: Claude <noreply@anthropic.com>
Releases 1.1.0 and 1.0.0post1 used a different underline than the older
releases, so everything from 1.0.0 down nested under 1.0.0post1.

Co-Authored-By: Claude <noreply@anthropic.com>
The start page included the README below a table of contents, and
several of its examples no longer ran (Sequence takes a list, Type is
now X690Type). The new structure:

- start page: what x690 is, install, an example, features, scope and
  project status
- usage: encoding, decoding single values and streams, enforce_type,
  strict and pretty()
- custom types: reusing an existing type and writing decode_raw and
  encode_raw, built on an SNMP GetRequest
- supported types: which universal types convert to which Python value
- API reference: one hand-written page per module instead of
  sphinx-apidoc output, so the CI no longer generates it

The module docstrings of x690 and x690.types shrink to short summaries
that point to these pages, and the glossary drops stale puresnmp
references.

Co-Authored-By: Claude <noreply@anthropic.com>
The detailed sections moved to the documentation. The Sequence examples
in the old README raised TypeError.

Co-Authored-By: Claude <noreply@anthropic.com>
pytest now collects the .rst pages under docs/ and README.rst. A
fixture restores the type registry after each page, so the custom types
a page defines do not leak into other tests.

Co-Authored-By: Claude <noreply@anthropic.com>
Break imports, constructors and byte strings over several lines so the
examples stay readable without scrolling, and note that pythonize()
converts the items of a sequence while value keeps x690 objects.

Co-Authored-By: Claude <noreply@anthropic.com>
The new version colours code with the design system's code tokens.

Co-Authored-By: Claude <noreply@anthropic.com>
x690 is no longer in maintenance mode: context-specific tags and Python
values for more universal types are planned. The README, contributing
guide and changelog say so, and the package is classified as
Production/Stable instead of Mature.

Co-Authored-By: Claude <noreply@anthropic.com>
Decode the same SNMP request with pyasn1, asn1tools, asn1crypto, pycrate
and x690, and compare where each takes the structure from, what it
returns and what a type checker sees. Every example was run against
pyasn1 0.6.4, asn1tools 0.169.0, asn1crypto 1.5.1 and pycrate 0.8.1.

The page explains the design decisions behind x690 and lists its current
limitations with their issues (#6 to #11). The start page links to it
and its status section reflects active development.

Co-Authored-By: Claude <noreply@anthropic.com>
benchmarks/ compares decode time and peak memory of x690 with pyasn1,
asn1tools, asn1crypto and pycrate on synthetic CDR-like data, in four
workloads: a stream of records, a whole document read by iteration, only
its last record, and a whole document read by index. The last one shows
the quadratic indexing of #16.

Run it with 'uv run --group bench python benchmarks/run_all.py'. It
writes a JSON report with the machine, library versions and x690 commit;
benchmarks/report.py renders one or more reports as tables. Memory is
measured on top of the loaded input, exactly on Linux and from
ru_maxrss elsewhere.

Co-Authored-By: Claude <noreply@anthropic.com>
report.py now starts with the system, CPU, memory, Python, library
versions, x690 commit and memory limit of every report it renders. The
JSON report already held these; the tables did not show them.

Co-Authored-By: Claude <noreply@anthropic.com>
Run on a 2-core, 3.7 GB machine with a 2 GB limit per case, at commit
cea5d6d. x690 needed 409 s to index 10,000 records (#16).

Co-Authored-By: Claude <noreply@anthropic.com>
A decoded Sequence re-decoded its whole item list on every len(), index
or iteration, so reading every item by index was quadratic: 200 items
took 40,200 header reads.

The sequence now records the start of each item once, in a compact
array, and decodes items on demand. Iterating no longer builds the
list of all items first, so the items read can be freed as the loop
goes on. decode_raw no longer copies the whole value to check whether
it is empty.

Closes #16

Co-Authored-By: Claude <noreply@anthropic.com>
X690Type declared __slots__, but its subclasses did not, so every value
still carried a __dict__, and bounds was assigned per instance outside
the slots. A decoded Integer took 472 bytes; it now takes 120.

bounds is a slot initialised in __init__. Custom types that declare no
slots keep working with a __dict__ of their own.

Closes #17

Co-Authored-By: Claude <noreply@anthropic.com>
Same machine and limits as the before report, at commit 62cddca. At
1,000,000 records x690 reads a whole document in 22 s with 8 MB on top
of the input (was 32 s and 558 MB), the last record in 1.6 s (was
7.2 s), and every record by index in 22 s (10,000 records took 409 s).

Co-Authored-By: Claude <noreply@anthropic.com>
The choosing page gains a performance section: decode time and peak
memory of all five libraries at 1,000,000 records, on the constrained
machine of the committed reports. The design notes explain that a
sequence records where its items start and creates items on access.
The changelog lists the sequence and slots changes and the benchmark.

Co-Authored-By: Claude <noreply@anthropic.com>
@exhuma exhuma self-assigned this Oct 26, 2026
@exhuma exhuma changed the title perf: decode large documents in constant memory, with a benchmark perf: decode large documents with a fraction of the memory, with a benchmark Oct 26, 2026
exhuma and others added 4 commits October 26, 2026 17:01
The performance section now leads with a run on a 6-core desktop with
31 GB of memory, where every library finishes: x690 reads a whole
document of 1,000,000 records in 10 s with 8 MB on top of the input,
against 13 s and 1.4 to 2 GB for asn1tools and asn1crypto, and 70 s and
146 s with 5.5 and 9.6 GB for pycrate and pyasn1. The small machine
remains as the memory-limited case.

Co-Authored-By: Claude <noreply@anthropic.com>
The start page and README now say that x690 decodes documents of a
million values in a few megabytes, and the 'No copies' feature becomes
'Large documents' with the benchmark numbers and a link to them. The
choosing page no longer says speed is not a selling point: x690 is the
fastest for whole documents, asn1crypto for record streams. It also
states the memory growth per item and lists limited memory as a reason
to choose x690.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Base automatically changed from docs/design-system-landing-page to main October 26, 2026 16:18
main holds #15 as one squash commit whose tree is identical to 3bd9a71,
the base of this branch, so this branch's files already contain it.

Co-Authored-By: Claude <noreply@anthropic.com>
@exhuma
exhuma merged commit cd3d9fb into main Oct 26, 2026
22 checks passed
@exhuma
exhuma deleted the perf/large-document-decoding branch October 26, 2026 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Declare __slots__ on all types to reduce memory per value Indexing a decoded Sequence is quadratic

1 participant