Skip to content

feat(bench): parsec-xml had no numbers against libxml2 or anyone - #200

Open
jamesconwaydev wants to merge 3 commits into
xml-conformancefrom
xml-bench
Open

jamesconwaydev wants to merge 3 commits into
xml-conformancefrom
xml-bench

Conversation

@jamesconwaydev

Copy link
Copy Markdown
Collaborator

Fixes #172.

Stacked on #199.

Problem. parsec-xml had no numbers. Nothing said how it compares with libxml2, which most XML on the web goes through (lxml, nokogiri, PHP), or with roxmltree and quick-xml, the Rust parsers people pick today. Running the comparison also showed a real cost: on 7,447 Material Design icons, about 450 bytes each, Parsec ran at 269 MB/s to roxmltree's 446, because every document built three hash maps and interned the built-in names before reading a byte, and its text buffer grew by doubling.

Change.

  • bench/libxml2, a submodule at v2.15.4, built by CMake as a Release static library without iconv, zlib, ICU or Python. bench/src/libxml2.rs declares the calls it needs (xmlReadMemory with XML_PARSE_NONET, xmlFreeDoc) and routes libxml2's allocations through the benchmark's allocator with xmlMemSetup, so its peak memory is counted as Parsec's is.
  • bench/xml.txt and bench/fetch-xml.sh: 102 MB of real XML pinned by SHA-256. Sitemaps (MDN, New York Times) and RSS and Atom feeds (BBC, Hacker News, the Rust blog, rust-lang/rust releases, r/programming) as raw Internet Archive snapshots; SVGs from a Wikimedia snapshot and two versioned npm packages; the XML spec's XML source; and the first 24 MB of the 1 November 2026 English Wikipedia dump, decompressed and cut after its last whole </page>.
  • cargo run --release --manifest-path bench/Cargo.toml -- xml times Parsec, libxml2 and roxmltree building trees, and quick-xml reading every event with namespaces resolved and every value normalized (it builds no tree, and the table says so). MB/s and peak MB per document, then the total.
  • perf(xml): the built-in prefixes and URIs get fixed ids with no map entries, a document's first 16 names are found by comparing rather than hashing, and the text buffer starts at the input's size. The icons go from 269 to 514 MB/s, and the Wikipedia slice from 1,275 MB/s and 183 MB peak to 1,449 MB/s and 100 MB.

Results. Apple M2 Max, macOS 26.7, Rust 1.100, today, MB/s · peak MB:

Document Size Parsec libxml2 roxmltree quick-xml (events)
MDN sitemap 1.94 MB 580 · 8.2 370 · 11.9 387 · 6.4 532
NYT news sitemap 0.68 MB 898 · 2.3 315 · 3.9 426 · 2.5 591
BBC News RSS 0.03 MB 648 · 0.1 250 · 0.2 405 · 0.1 628
Hacker News RSS 0.02 MB 636 · 0.0 274 · 0.1 526 · 0.1 810
Rust blog Atom 0.10 MB 669 · 0.1 294 · 0.2 305 · 0.1 481
rust-lang/rust releases Atom 0.10 MB 623 · 0.1 257 · 0.2 300 · 0.2 482
r/programming Atom 0.03 MB 602 · 0.1 246 · 0.1 326 · 0.1 480
World map SVG 1.10 MB 1,594 · 1.7 316 · 3.5 952 · 0.9 454
Bootstrap Icons sprite 1.10 MB 1,151 · 2.0 260 · 4.3 682 · 1.2 380
Material Design Icons, 7,447 files 3.35 MB 514 · 12.8 140 · 24.2 444 · 5.0 308
The XML spec 0.21 MB 424 · 1.1 181 · 2.0 185 · 1.2 refused
Wikipedia dump slice 93.64 MB 1,449 · 100.2 566 · 146.2 319 · 188.9 1,565
All 102.28 MB 1,314 495 327

Parsec is 2.66 times libxml2's speed over the whole set and four times roxmltree's, ahead of both on every document, and holds no more memory than libxml2 on any. quick-xml is ahead on the Wikipedia slice and the Hacker News feed, and level on the BBC feed, while doing less: no tree, nothing kept. It refuses the XML spec because it can't expand entities its DTD declares. A second run after the perf change agreed within 2% on most documents. The small feeds and the MDN sitemap moved by up to 8%, and on the BBC feed quick-xml came out ahead, 628 to 608. The load was around 4 from other builds.

Tests. The bench job in CI builds it, libxml2 included, and lints it with clippy, rustfmt and cargo-deny (quick-xml and roxmltree are MIT). The real documents aren't in the repository, so CI doesn't run the XML table. cargo test -p parsec-xml passes after the perf change, with a new test for names found either side of the 16-name switch, and xmlconf still has 1,915 passing.

Wiki patch: Benchmarks

In the first paragraph, after the CSS sentence, add:

On 102 MB of real XML it parses 2.66 times as fast as libxml2 and four times as fast as roxmltree.

Add this section after ## CSS and its subsections, before ## The final boss:

## XML

Parsec's `parsec-xml` ([XML](XML)) against [libxml2](https://gitlab.gnome.org/GNOME/libxml2), [roxmltree](https://github.com/RazrFalcon/roxmltree) and [quick-xml](https://github.com/tafia/quick-xml), on 102 MB of real XML. Over all of it Parsec parses 1,314 MB/s, 2.66 times libxml2's 495 and four times roxmltree's 327. It's ahead of both on every document, and holds no more memory than libxml2 on any: 100 MB for the Wikipedia slice, where libxml2 holds 146.

- **libxml2 2.15.4** comes in as a submodule at its release tag and is built by CMake as a Release static library, without iconv, zlib, ICU or Python. The benchmark calls `xmlReadMemory` with `XML_PARSE_NONET` and its other defaults, limits included, and frees with `xmlFreeDoc`. Its `malloc`, `realloc`, `strdup` and `free` go through the benchmark's allocator with `xmlMemSetup`, so its memory is counted as Parsec's is.
- **roxmltree 0.21.2** builds its tree, with `allow_dtd` on so the XML spec's `<!DOCTYPE>` is allowed.
- **quick-xml 0.42.0** builds no tree: it's a pull parser. It's timed reading every event through `NsReader`, namespaces resolved, with every text and attribute value normalized and references resolved, which is the least a program reading the document has to do. It can't expand entities a DTD declares, so it refuses the XML spec.
- **Parsec** is `parsec_xml::parse(text)`, which builds the tree with every name resolved, a span on every node and attribute, and entities expanded.

Parsec and roxmltree get the text as a `&str`; libxml2 gets the same bytes and checks the UTF-8 itself. The timing is the HTML table's: the median of at least 30 runs after three warm-ups (five for a parse over a second), peak memory the heap's growth while one document is built and held. A set (the icons) is every file parsed and held, one after another.

**The documents** are pinned by SHA-256 in [`bench/xml.txt`](https://github.com/parsec-html/parsec/blob/main/bench/xml.txt), and `bench/fetch-xml.sh` downloads them into `bench/corpus/xml/`: sitemaps and feeds as raw Internet Archive snapshots, SVGs from a Wikimedia snapshot and two versioned npm packages, the XML spec's own XML source, and the first 24 MB of the 1 November 2026 English Wikipedia dump (`pages-articles1`), decompressed and cut after its last whole page. That dump leaves dumps.wikimedia.org after a few months; to move to a newer one, change its address in `xml.txt` and take the new hash from `bench/fetch-xml.sh --print`.

MB/s, then peak MB:

| Document | Size | Parsec | libxml2 | roxmltree | quick-xml (events) | Parsec vs libxml2 |
|---|---:|---:|---:|---:|---:|---:|
| MDN sitemap | 1.94 MB | 580 · 8.2 | 370 · 11.9 | 387 · 6.4 | 532 | 1.57× |
| New York Times news sitemap | 0.68 MB | 898 · 2.3 | 315 · 3.9 | 426 · 2.5 | 591 | 2.85× |
| BBC News RSS | 0.03 MB | 648 · 0.1 | 250 · 0.2 | 405 · 0.1 | 628 | 2.59× |
| Hacker News RSS (hnrss) | 0.02 MB | 636 · 0.0 | 274 · 0.1 | 526 · 0.1 | 810 | 2.32× |
| Rust blog Atom | 0.10 MB | 669 · 0.1 | 294 · 0.2 | 305 · 0.1 | 481 | 2.28× |
| rust-lang/rust releases Atom | 0.10 MB | 623 · 0.1 | 257 · 0.2 | 300 · 0.2 | 482 | 2.42× |
| r/programming Atom | 0.03 MB | 602 · 0.1 | 246 · 0.1 | 326 · 0.1 | 480 | 2.44× |
| World map SVG (Wikimedia) | 1.10 MB | 1,594 · 1.7 | 316 · 3.5 | 952 · 0.9 | 454 | 5.04× |
| Bootstrap Icons sprite | 1.10 MB | 1,151 · 2.0 | 260 · 4.3 | 682 · 1.2 | 380 | 4.43× |
| Material Design Icons, 7,447 files | 3.35 MB | 514 · 12.8 | 140 · 24.2 | 444 · 5.0 | 308 | 3.66× |
| The XML spec, in XML | 0.21 MB | 424 · 1.1 | 181 · 2.0 | 185 · 1.2 | refused | 2.34× |
| Wikipedia dump slice | 93.64 MB | 1,449 · 100.2 | 566 · 146.2 | 319 · 188.9 | 1,565 | 2.56× |
| **All documents** | **102.28 MB** | **1,314** | **495** | **327** | | **2.66×** |

These are an Apple M2 Max (12 cores, 32 GB), macOS 26.7, Rust 1.100, on 21 November 2026, with the one-minute load around 4 from other builds. A second run gave the same within 2% on most documents; the small feeds and the MDN sitemap moved by up to 8%, and on the BBC feed quick-xml came out ahead, 628 MB/s to 608.

Where quick-xml is ahead, the Wikipedia slice and the Hacker News feed, and where it's level, the BBC feed, it's doing less: it builds no tree and keeps nothing. roxmltree holds less memory than Parsec on the SVGs and the MDN sitemap, because its tree points into your text where Parsec copies the text once, unescaped, so its tree outlives the input. On the Wikipedia slice it's the other way: roxmltree holds 189 MB to Parsec's 100.

<details>
<summary>The raw output</summary>

```text
Lexbor 3.0.1, html5ever 0.40.1, Ada 4.0.0, Parsec 679594f; macos aarch64, 12 threads available

XML: parsec-xml, libxml2 2.15.4, roxmltree 0.21.2, quick-xml 0.42.0 (events only, no tree)

MB/s, peak MB                       size            Parsec           libxml2         roxmltree         quick-xml  vs libxml2
  sitemap-mdn                    1.94 MB      580      8.2      370     11.9      387      6.4      532      0.0       1.57x
  sitemap-nytimes-news           0.68 MB      898      2.3      315      3.9      426      2.5      591      0.0       2.85x
  rss-bbc-news                   0.03 MB      648      0.1      250      0.2      405      0.1      628      0.0       2.59x
  rss-hacker-news                0.02 MB      636      0.0      274      0.1      526      0.1      810      0.0       2.32x
  atom-rust-blog                 0.10 MB      669      0.1      294      0.2      305      0.1      481      0.0       2.28x
  atom-rust-releases             0.10 MB      623      0.1      257      0.2      300      0.2      482      0.0       2.42x
  atom-reddit-programming        0.03 MB      602      0.1      246      0.1      326      0.1      480      0.0       2.44x
  svg-world-map                  1.10 MB     1594      1.7      316      3.5      952      0.9      454      0.0       5.04x
  svg-bootstrap-icons            1.10 MB     1151      2.0      260      4.3      682      1.2      380      0.0       4.43x
  svg-mdi-icons                  3.35 MB      514     12.8      140     24.2      444      5.0      308      0.2       3.66x
  xml-spec                       0.21 MB      424      1.1      181      2.0      185      1.2           refused       2.34x
  wikipedia-dump                93.64 MB     1449    100.2      566    146.2      319    188.9     1565      0.0       2.56x
  all documents                102.28 MB     1314               495               327                          -       2.66x
```

</details>

To run only this table: `bench/fetch-xml.sh`, then `cargo run --release --manifest-path bench/Cargo.toml -- xml`. It needs the `bench/libxml2` submodule, CMake and a C compiler.

On 7,447 Material Design icons, about 450 bytes each, parsec-xml ran at
269 MB/s to roxmltree's 446: each document built three hash maps and
interned the built-in prefixes and namespaces before reading a byte,
and its text buffer grew by doubling.

The built-in prefixes and URIs now have fixed ids and no entries, a
document's first 16 names are found by comparing rather than hashing,
and the text buffer starts at the input's size. The icons parse at 514
MB/s, and a 94 MB Wikipedia slice goes from 1,275 MB/s and 183 MB peak
to 1,449 MB/s and 100 MB.
bench/fetch-xml.sh downloads 102 MB of real XML pinned by SHA-256:
sitemaps and feeds from the Internet Archive, SVGs, the XML spec, and a
slice of a dated Wikipedia dump. `-- xml` times Parsec against
libxml2's C, built from a submodule at v2.15.4 with its allocations
counted, roxmltree and quick-xml, giving MB/s and peak memory.

Parsec parses 1,314 MB/s to libxml2's 495 and roxmltree's 327, ahead of
both on every document. quick-xml, timed reading events without a tree,
is ahead on the Wikipedia slice and the Hacker News feed.
Brings the branch onto its new base after the pull request below it merged.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

xml The XML parser, parsec-xml

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant