Repository navigation
Conversation
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. Core Committers: Use this line as a base for the props when committing in SVN: At least one To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. Props Bot commandsYou can trigger Props Bot actions by commenting on this PR:
|
sirreal
force-pushed
the
html-api/benchmark
branch
2 times, most recently
from
October 12, 2026 16:15
e9c617f to
f834251
Compare
A PHP CLI that times WP_HTML_Tag_Processor and WP_HTML_Processor parsing (next_token() to the end of the document) over a corpus of real public pages and seeded synthetic documents, and compares two checkouts with interleaved samples, fresh worker processes per round and confidence intervals on the ratio of medians. - bench.php orchestrates one worker per checkout and prints a table, markdown or JSON report; compare.php compares two saved runs. - worker.php loads one checkout's HTML API with function stubs and runs timed jobs over a JSON line protocol. - lib/class-benchmark-synthetic.php generates 13 document shapes (dense tags, long text, entities, UTF-8, comments, attributes, deep nesting, script and style, whitespace, foreign content, formatting, tables, a WordPress post), byte-identical for a given seed across PHP versions. - corpus/manifest.json names twenty public pages with sizes and hashes; fetch-corpus.php restores them into the gitignored corpus/real/. An A/A run with the defaults (4 rounds of 10 samples) reports no difference on all 64 rows. The README documents the method and why the round count defaults to an even number.
…e, which this repository uses instead of nested ignore files.
…the JIT, and serialize timed runs. Under the tracing JIT, the code paths a process compiles first set the speed of everything it runs afterwards: a worker that parsed wordpress.org's home page first ran it and later documents 10% to 16% slower than one that started with another document. With one fixed order every round shared that history, so the interval over rounds did not include it. Each round now runs the (document, parser) pairs in an order shuffled from the seed and round number, the same for both trees, and the saved JSON records the orders. The HTML Processor needs about six parses before its traces settle. With --jit the warmup is now at least 10 parses and 200 ms (--warmup-ms), which took its CV on wporg-home from 12% to 3%. bench.php now holds a lock on /tmp/wp-html-api-bench.lock for the whole run and waits while another run holds it; with-lock.php runs any other command under the same lock, for profiling and ad hoc timing. A worker with a profiler extension loaded stops the run.
…en two checkouts. A performance change must not change what the parser reports. difftrace.php loads the base and head Tag Processors into one process under renamed classes and compares the return value, every public accessor and every _doing_it_wrong() call after every step of six walks: next_token(), next_tag(), next_tag() visiting closers, next_tag() with queries, with modifications, and with bookmarks and seeks. For the HTML Processor it compares a hash of the token stream per input, one process per checkout. Inputs are the corpus, the synthetic shapes at two sizes, edge cases with every prefix, and 20,000 mutated corpus slices. Base against itself reports no differences over 4.6 million steps; dropping "\r" from the whitespace skipped around "=" or the XMP case from the special-element switch each produce hundreds of differing inputs.
…sults. difftrace.php gains a walk that mixes modifications with bookmarks and seeks, special-element edge cases (case, near-miss names, NUL bytes, leading newlines, virtual closers), and an HTML Processor pass that calls get_updated_html() and then sets an attribute while walking, so that an update is pending when the processor parses a virtual token or an error again. It now finds the eight inputs on which matching special elements by raw bytes, instead of through get_tag(), changes HTML Processor output. get_updated_html() calls are capped at 20 per input, since each one copies the whole document. proc_open() no longer receives STDOUT or STDERR: PHP seeks a redirected file's shared offset to the stream's own position before handing it on, and the outputs overwrote each other. The report prints each round's overall change under the overall line, so one slow round is visible instead of only widening the interval.
sirreal
force-pushed
the
html-api/benchmark
branch
from
October 29, 2026 05:37
f834251 to
8422879
Compare
This was referenced Oct 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a parsing benchmark for the HTML API under
tests/performance/html-api/, so a change toWP_HTML_Tag_ProcessororWP_HTML_Processorcan be measured against a baseline checkout with a confidence interval instead of a pair of wall-clock numbers.The timed loop constructs a processor and calls
next_token()until it returns false, for the Tag Processor and for the HTML Processor throughcreate_full_parser(). Nothing else is timed: noget_tag(), no attribute reads, no modifications. A document the HTML Processor cannot finish is still timed but reported separately and left out of the aggregate.How a run works:
bench.phpstarts oneworker.phpprocess per checkout. Each worker loads that checkout'ssrc/wp-includes/html-api/with stubs for the few WordPress functions it calls (__(),esc_url(),wp_kses_uri_attributes()and similar), so WordPress itself is not loaded and--basemay point at a checkout that predates this tool.faster,slower,no difference), and the coefficient of variation of each tree's samples; then a geometric-mean overall line per parser with its own interval. Output is a table, markdown or JSON;--savekeeps every sample, andcompare.phpcompares two saved runs.Documents:
lib/class-benchmark-synthetic.phpgenerates 13 shapes at a target size (default 200 KB) from a seed: dense tags, long text, entities, multi-byte text, many comments, attribute-heavy tags, deep nesting, script and style bodies, whitespace runs, inline SVG and MathML, misnested formatting, tables, and a block-themed WordPress post. Output is byte-identical for a given seed on PHP 8.4 and 8.5, and every shape parses to the end on trunk's HTML Processor. Thetablesandformatting-adoptionshapes stay inside what trunk supports, since foster parenting and the general adoption agency case still bail.corpus/manifest.jsonnames 20 public pages with sizes and hashes (the HTML standard's parsing section, the W3C HTML5 syntax chapter, RFC 9110, seven Wikipedia articles in four scripts pinned by revision, five WordPress-rendered pages, an MDN reference, a php.net manual page, Hacker News), plus the 15 MB single-page HTML standard behind--include-optional.fetch-corpus.phprestores them into the gitignoredcorpus/real/; nothing third-party is committed.--corpus <dir>reads any directory of HTML files.One measurement result shaped the defaults. Two PHP processes running identical code differ by a constant 0.3% to 2% for the life of the process, and the second-started one is the slower, so interleaving within a process pair cannot cancel it: a one-round A/A run reported
slowerat +1.8% on every HTML Processor row with CVs of 0.2%. The tool therefore repeats the schedule over--roundsfresh worker pairs and takes a t-interval over the per-round ratios; the default is 4 rounds of 10 samples because an odd count leaves the start order unbalanced (3 rounds gave 8 of 64 rowsslower, all the same sign). With the defaults, an A/A run over all 32 documents reportsno differenceon all 64 rows, with both overall intervals centred on zero; per-row resolution is about ±1%, and a full run takes about 34 minutes on an M-series laptop.As a first use, replacing the two length-1
strspn()set checks in the Tag Processor's parse loop withstrpos()on the set measured -1.2% [-2.3%, -0.2%] on the Tag Processor and -0.6% [-1.5%, +0.4%] on the HTML Processor, every point estimate negative, with identical token output. That change is not in this PR.Limitations: the WHATWG pages are fetched unpinned because the site's robots.txt disallows
/commit-snapshots/, and the Wikipedia pages carry per-request values so their hashes are informational;fetch-corpus.phpreports drift on unpinned entries rather than failing.SPEC.mdis the design document the tool was built from and can be dropped before any upstream PR.Trac ticket: none yet; this is a fork PR for review of the tool itself. trac#65967 (the HTML API fuzzing ticket) is the nearest existing ticket.
Use of AI Tools
AI assistance: Yes
Tool(s): Claude Code
Model(s): Claude Fable 5.1
Used for: Implementation, corpus selection, synthetic document generator, documentation and the validation runs, from a design settled in a question round with the author. The author has not yet reviewed the code line by line; the A/A validation and the strspn experiment were run and read by the agent.
This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.