Repository navigation
feat(reduce): a page that broke Parsec couldn't become a test - #211
Open
jamesconwaydev wants to merge 3 commits into
Open
jamesconwaydev wants to merge 3 commits into
jamesconwaydev wants to merge 3 commits into
Conversation
When a real site's page breaks Parsec, the page is the site's and can't go in the repository, and finding the few bytes that matter by hand takes an afternoon. crates/parsec-reduce is a binary, not published, that shrinks a page by delta debugging: ddmin over tags and text runs, then attributes, then words, then characters, round after round until nothing more goes. `--check` says what failing is: `panic` (at the page's own panic location), `hang:MS`, `differs-from-html5ever` (html5lib tree dumps, scripting off) or `cmd:COMMAND`. Each candidate runs in a child process, so an abort or a hang can't stop the reducer. Fixtures go in crates/parsec-html/tests/regressions/, each starting with a site, date, failure and issue comment. tests/regressions.rs parses every one inside 10 s without a panic, and matches a `.tree` beside it when there is one; it refuses files over 64 KB or without the header. CONTRIBUTING.md has the policy: reduced inputs only, never whole pages.
# Conflicts: # Cargo.lock # Cargo.toml
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #203.
Problem. When a real page breaks Parsec, the page belongs to the site and can't go in the repository. Getting from a 1 MB page to the few bytes that matter was done by hand, and nothing held a found bug down afterwards: there was no place for real-web regressions and no test that ran them.
Change.
crates/parsec-reduce, a workspace binary withpublish = false:cargo run --release -p parsec-reduce -- page.html --check <panic|hang:MS|differs-from-html5ever|cmd:COMMAND> [-o OUT]. It runs ddmin over tags and text runs first, then attributes, then words of text, then characters, and repeats the four until a round removes nothing. It's deterministic, prints the size after each pass, and writespage.reduced.html(or-o).panicholds every candidate to the page's own panic location (or a stack overflow), so the reduction can't slide onto a different bug.differs-from-html5evercompares html5lib-tests tree dumps. Parsec's comes fromSerializeOptions::html5test; html5ever's (0.40, the versionbench/uses) is dumped insrc/tree.rs. Both parse with scripting off, and Parsec with declarative shadow roots off, since html5ever has none.cmd:COMMANDruns throughsh -cwith the candidate's path as$1, and counts as failing while it exits non-zero.--child), so an abort or a hang is killed and can't take the reducer down with it.crates/parsec-html/tests/regressions.rsparses everytests/regressions/*.htmlon its own thread with a 10 s budget. A panic or a slow parse fails it. So does a.treebeside the fixture that doesn't match, a file over 64 KB, or a missing<!-- site: URL; date: YYYY-MM-DD; failed: ...; issue: #N -->first line.BLESS_REGRESSIONS=1writes the.treefiles. It runs in CI withcargo test --workspace. The directory has no fixtures yet: the first real one comes with the first crawl failure (Netweir crawls production sites through Parsec and reports what breaks #204 onward).scripts/package-twice.shand CI'sreproduciblejob pass--exclude parsec-reduce, so a release never builds or attaches a package of it. Its dependency onparsec-html-corecarriesversion = "0.2.0", whichcargo package --list(the test-data check) needs. That makes it one morecrates/*/Cargo.tomlrequirement for the version bump, which the Releasing page's glob already covers.crates/parsec-htmlnow excludestests/from its package, like the other crates, so CI's "No crate ships test data" check still passes.Trade-off. The page is read as UTF-8, and anything invalid is replaced. A bug in byte decoding can't be reduced with the built-in checks.
cmd:with your own decoding check can, and the limit is noted inmain.rs. html5ever is now in the workspace'sCargo.lock, but onlyparsec-reduceuses it.Seeded bug. A generated shop page of 1,000,087 bytes (cards, inline style and script, a header nav), with one
<aside><nav><ul><li class=deals><table>buried at card 3,141. The "bug" is a Pythonhtml.parserscript that exits 1 when a<table>starts inside an<li>inside a<nav>:1 MB down to 16 bytes, which is 1-minimal: no single character more can go. I also ran
--child differsover every seventh page of the final boss batch (the UTF-8 ones) and over the shop page, and Parsec's and html5ever's dumps matched on all of them. Sodiffers-from-html5everdoesn't fire on pages where the two agree.Tests.
reduce::tests:tags_cover_the_input,attributes_of_start_tags_only,words_of_text_onlyandchars_are_whole_characterscover each split (they fail if a split drops or overlaps bytes).ddmin_keeps_only_what_failschecks ddmin keeps exactlychofabcdefghijin under 40 tests, andddmin_with_nothing_to_removecovers the empty and already-minimal cases.reduce_finds_the_nested_structurereduces a 300-row page to<li<tabletwice, with the same result both times.tree::tests:the_dumps_agree_where_the_parsers_docovers doctypes, templates, SVG and MathML with xlink attributes, comments and noscript, and fails if the html5ever dump drifts from html5lib's format.the_dump_is_html5lib_spins the exact text.maintests: check parsing (badhang:/cmd:rejected), panic signatures, the output path.tests/cli.rs: the--childmodes,a_page_that_doesnt_fail_is_refused(exit 1 underpanic,hang:5000anddiffers-from-html5ever),bad_arguments(exit 2), anda_command_check_reduces_a_page, a 128 KB page reduced throughcmd:to<li<table(Unix only, as it needssh).regressions.rs:every_fixture,headers(seven malformed headers, each with its reason), anda_fixture_is_held_to_its_tree(bless, match, mismatch, no header, over 64 KB).Passed locally:
cargo test --workspace,cargo clippy --workspace --all-targets -- -D warnings,cargo fmt --all --check,cargo docfor both crates with-D warnings,cargo deny check licenses bans sourcesandscripts/check-headers.sh.