Skip to content

feat(bench): parsec-js had no numbers against oxc or swc - #191

Open
jamesconwaydev wants to merge 2 commits into
js-limitsfrom
js-bench
Open

jamesconwaydev wants to merge 2 commits into
js-limitsfrom
js-bench

Conversation

@jamesconwaydev

Copy link
Copy Markdown
Collaborator

Fixes #156. Fixes #152.

Builds on #189, which this is based on; once that merges, this is rebased onto its base.

Problem. Nothing timed parsec-js. Whether it was quick, and how it stood against oxc_parser and swc_ecma_parser, which are what a Rust project would reach for, was a guess. #152's last check was waiting too: that the ESTree output matches acorn's on the bench corpus.

Change.

  • bench/src/js.rs, run with parsec-html-bench js: parsec-js, oxc_parser 0.157.0 and swc_ecma_parser 47.0.0 (with swc_common 27.0.0), pinned with = in bench/Cargo.toml, on the same strings. Each builds its whole tree and keeps it until the clock stops. A time is the median of at least five runs after a warm-up, more for small sources. Peak memory comes from the counting allocator the HTML tables use.
    • Parsec and oxc keep comments; swc is given no comment store, so it keeps none.
  • The corpus. bench/scripts.txt pins react-dom 19.2.0's production client, three.js 0.181.0 (a module) and typescript.js 5.9.3 on jsDelivr, with their SHA-256, and bench/fetch-pages.sh fetches them with the pages and stylesheets. The inline <script> elements of each real page are found with Parsec's own selectors, one source per page, each script parsed on its own. One any parser rejects would be left out and counted; none was.
  • parsec-html-bench js-dump DIR writes every source out, and crates/parsec-js/scripts/compare-acorn.sh --bench uses it to compare each tree with acorn 8.19.0's. That's the check The AST: arena nodes, spans, ESTree JSON out #152 asked for. It's a script, not a CI step: it needs the fetched corpus and Node.
  • parsec-js parses at 0.57 times oxc's speed #190 is the gap this found: oxc is faster.
  • README's JavaScript row and ROADMAP's M10 have the numbers.

Results. Apple M2 Max, macOS, Rust 1.100, load average 6.5:

Parsec MB/s oxc MB/s swc MB/s Parsec peak oxc peak swc peak
react-dom 19.2.0 198 290 140 2.6 MB 4.2 MB 5.1 MB
three.js 0.181.0 223 393 190 3.0 MB 4.2 MB 5.4 MB
typescript.js 5.9.3 199 347 157 44.1 MB 67.1 MB 90.5 MB
All 11.6 MB 202 356 160

Parsec is 1.26 times swc's speed and 0.57 times oxc's, and holds the least memory on every source. An earlier run gave the same totals within 2%.

Tests.

  • compare-acorn.sh --bench: 107 sources (3 bundles, 104 inline scripts), all the same as acorn's.
  • The CI bench job's steps pass locally: cargo fmt --check, cargo clippy --release -D warnings and cargo deny check licenses bans sources on bench/Cargo.toml, and the "shop 1 MB" "unclosed tables" run. The oxc and swc trees are MIT and Apache-2.0, and deny.toml takes them with no changes.

The wiki. This is for the Benchmarks page once this merges; I'll push it to the wiki then.

Benchmarks.md patch
diff --git a/Benchmarks.md b/Benchmarks.md
index 72b01ac..af789f6 100644
--- a/Benchmarks.md
+++ b/Benchmarks.md
@@ -1,4 +1,4 @@
-Parsec against Lexbor's own C and html5ever, in one process, on the same strings. On 13 real pages Parsec parses 1.5 times as many megabytes a second as Lexbor, is ahead on every page, and holds less memory on each. Over the same pages its selectors are 3.2 times as fast as Lexbor's. On the links in those pages, its URL parser is 2.28 times as fast as Lexbor's and 1.33 times as fast as Ada's. On real stylesheets it tokenizes 1.85 times as fast as Lexbor's CSS module and parses 1.9 times as fast, and parses 1.08 times as fast as Servo's cssparser. It's ahead on all eight hostile inputs too, and they're in the tables below with the rest.
+Parsec against Lexbor's own C and html5ever, in one process, on the same strings. On 13 real pages Parsec parses 1.5 times as many megabytes a second as Lexbor, is ahead on every page, and holds less memory on each. Over the same pages its selectors are 3.2 times as fast as Lexbor's. On the links in those pages, its URL parser is 2.28 times as fast as Lexbor's and 1.33 times as fast as Ada's. On real stylesheets it tokenizes 1.85 times as fast as Lexbor's CSS module and parses 1.9 times as fast, and parses 1.08 times as fast as Servo's cssparser. Its JavaScript parser is 1.26 times as fast as swc's on real bundles and 0.57 times as fast as oxc's, with less memory than either. It's ahead on all eight hostile inputs too, and they're in the tables below with the rest.
 
 > [!NOTE]
 > These are one machine's numbers: an Apple M2 Max (12 cores, 32 GB), macOS 26.7, Rust 1.100, on 15 November 2026, Parsec at the commit that added `bench/` for the parse tables, and with netweir-html at 6034a8e synced in for the selector table. Run it on yours: it takes about five minutes.
@@ -274,6 +274,54 @@ The same run then styles each of the 13 real pages: parsed, then every element's
 
 To run only this table: `cargo run --release --manifest-path bench/Cargo.toml -- css`. It needs the real pages and the stylesheets from `bench/fetch-pages.sh`.
 
+## JavaScript
+
+Parsing JavaScript to a syntax tree, against [oxc_parser](https://crates.io/crates/oxc_parser) 0.157.0 and [swc_ecma_parser](https://crates.io/crates/swc_ecma_parser) 47.0.0, run with `parsec-html-bench js`. The corpus is 11.6 MB: the three bundles pinned in [`bench/scripts.txt`](https://github.com/parsec-html/parsec/blob/main/bench/scripts.txt) (react-dom 19.2.0's production client, three.js 0.181.0 as a module, and typescript.js 5.9.3), fetched from jsDelivr by `bench/fetch-pages.sh` and checked against their SHA-256, and the inline `<script>` elements of each real page, parsed one by one.
+
+- **Parsec** is `parsec_js::parse`, which keeps the comments beside the tree.
+- **oxc** is `Parser::new(&allocator, src, SourceType::cjs())` (`mjs()` for the module) and `.parse()`, with a fresh `Allocator` each time, kept with the program in it. It keeps comments too.
+- **swc** is `Parser::new(Syntax::Es(..), StringInput, None)` and `parse_script` or `parse_module`. With no comment store it keeps none, the least it can be asked to do.
+
+A time is the median of at least five runs after one warm-up, more for small sources, up to half a second's worth; freeing isn't timed. Peak memory is counted as for the HTML tables. An inline script any of the three rejects would be left out; none was.
+
+| | Parsec MB/s | oxc MB/s | swc MB/s | Parsec peak | oxc peak | swc peak | Parsec vs oxc | Parsec vs swc |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| react-dom 19.2.0, 0.54 MB | 198 | 290 | 140 | 2.6 MB | 4.2 MB | 5.1 MB | 0.68× | 1.41× |
+| three.js 0.181.0, 0.63 MB | 223 | 393 | 190 | 3.0 MB | 4.2 MB | 5.4 MB | 0.57× | 1.17× |
+| typescript.js 5.9.3, 9.11 MB | 199 | 347 | 157 | 44.1 MB | 67.1 MB | 90.5 MB | 0.57× | 1.27× |
+| Inline scripts, 1.30 MB | 123 to 232 | 254 to 672 | 87 to 201 | | | | 0.30× to 0.50× | 1.01× to 1.42× |
+| **All, 11.58 MB** | **202** | **356** | **160** | | | | **0.57×** | **1.26×** |
+
+Parsec is ahead of swc on every source and holds less memory than either on every one. oxc is faster on every one, most of all on the small inline scripts; closing that is [#190](https://github.com/parsec-html/parsec/issues/190). On 21 November 2026, at the commit that added the JavaScript bench, with the one-minute load at 6.5; a run before it gave the same totals within 2%.
+
+Every tree is the one acorn gives: `crates/parsec-js/scripts/compare-acorn.sh --bench` parses all 107 sources with acorn 8.19.0 and with Parsec and finds no difference.
+
+<details>
+<summary>The raw output</summary>
+
+```text
+JavaScript: oxc_parser 0.157.0, swc_ecma_parser 47.0.0. MB/s and peak MB, median of at least 5 runs
+                                          KB             Parsec                oxc                swc  vs oxc vs swc
+react-dom-19.2.0.js                    536.0       198      2.6       290      4.2       140      5.1   0.68x  1.41x
+three-0.181.0.mjs                      626.2       223      3.0       393      4.2       190      5.4   0.57x  1.17x
+typescript-5.9.3.js                   9112.6       199     44.1       347     67.1       157     90.5   0.57x  1.27x
+wikipedia-html <script>                  8.1       204      0.0       672      0.1       201      0.1   0.30x  1.01x
+wikipedia-population <script>            7.6       199      0.0       648      0.1       187      0.1   0.31x  1.06x
+mdn-html <script>                        0.3       195      0.0       409      0.0       144      0.0   0.48x  1.36x
+bbc-news <script>                        3.5       151      0.0       407      0.1       139      0.0   0.37x  1.09x
+stackoverflow <script>                   6.9       232      0.0       547      0.2       190      0.1   0.43x  1.23x
+python-stdtypes <script>                 1.3       177      0.0       476      0.0       147      0.0   0.37x  1.20x
+nytimes <script>                       982.7       223      5.7       450      6.4       182      6.9   0.50x  1.23x
+reddit-programming <script>             40.2       123      0.5       254      1.1        87      0.8   0.48x  1.42x
+ebay <script>                          252.4       202      1.5       592      2.4       195      1.9   0.34x  1.04x
+whatwg-parsing <script>                  0.2       185      0.0       388      0.0       134      0.0   0.48x  1.38x
+whatwg-full <script>                     0.2       185      0.0       388      0.0       139      0.0   0.48x  1.33x
+all                                  11578.2       202     44.1       356     67.1       160     90.5   0.57x  1.26x
+  ("vs" is Parsec's speed over the other's; 0 inline scripts left out because a parser rejects them)
+```
+
+</details>
+
 ## The final boss
 
 Everything above is one clean page at a time, already decoded. A crawler's day isn't like that. It gets 1,000 pages of whatever the web sent: scraps of broken markup, a page in Shift_JIS whose `<meta>` says UTF-8, and now and then 25 MB of table. The final boss is that day, built from a seed, fed to each parser as bytes, and timed per page with the process's peak memory. It answers a different question from the tables above: not how fast, but whether anything falls over, and what a whole batch costs.

Nothing timed the JavaScript parser, so whether it was fast, or how it
stood against the parsers people use, was a guess.

- `parsec-html-bench js` times parsec-js, oxc_parser 0.157.0 and
  swc_ecma_parser 47.0.0, pinned exactly, on the same strings: MB/s from
  the median of at least five runs, and peak memory from the counting
  allocator the HTML tables use.
- The corpus: react-dom 19.2.0, three.js 0.181.0 as a module and
  typescript.js 5.9.3, listed with their SHA-256 in bench/scripts.txt
  and fetched by fetch-pages.sh, and each real page's inline scripts.
- `parsec-html-bench js-dump DIR` writes every source out, and
  compare-acorn.sh --bench uses it to check each tree against acorn's.
- README's JavaScript row and ROADMAP's M10 have the numbers: 1.26 times
  swc's speed, 0.57 times oxc's, less memory than either.
Brings the branch onto its new base after the pull request below it merged.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bench Benchmarks against Lexbor and others javascript The JavaScript parser, parsec-js

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant