Skip to content

fix(js): invalid regular expression patterns parsed as if valid - #194

Open
jamesconwaydev wants to merge 2 commits into
js-scopefrom
js-regexp
Open

jamesconwaydev wants to merge 2 commits into
js-scopefrom
js-regexp

Conversation

@jamesconwaydev

@jamesconwaydev jamesconwaydev commented Nov 21, 2026 •

Copy link
Copy Markdown
Collaborator

Part of #159: the regular expression grammar. The rest of #159 is the next PR in this stack.

Builds on #193, which this is based on; once the stack below it merges, this is rebased onto its base.

Problem. The lexer finds where a regular expression literal ends and checks its flags, but nothing read the pattern between the slashes. /[z-a]/, /(?<a>x)(?<a>y)/, /a**/, /\p{Lettr}/u and /[a-z&&b]/v all parsed, where an engine throws a SyntaxError before any code runs. test262 skipped 358 parse negatives for it.

Change. A new module, parsec_js::regexp, with one function: validate(pattern, flags) -> Result<(), Diagnostic>. It reads the pattern by ECMA-262's grammar (22.2.1) and its early errors and gives the error a RegExp built from it would throw. It builds nothing and matches nothing.

  • Three modes, as the flags choose:
    • No u or v: Annex B's B.1.2. Legacy octal and identity escapes, \8 and \9, \c without a letter as a backslash, \k as k until a named group appears, a lookahead quantified, { and } as characters unless they form a quantifier with nothing before it (/{1}/ throws), [\d-a] as a union, and the pattern read as UTF-16 code units, so /[💩-💫]/ throws as it does in V8.
    • u: the strict grammar. Every escape has to be one, ], { and } have to be escaped, \1 has to name a group.
    • v: nested classes, && and -- (never mixed, never with a range as an operand), \q{…}, properties of strings, the escapable and double punctuators, and the MayContainStrings rule that keeps [^\p{RGI_Emoji}] and \P{RGI_Emoji} errors.
  • Groups: named groups with \u escapes and astral names; ES2025's duplicate names, allowed only in separate alternatives; \k<name> checked against every group, before or after it; modifiers (?i-ms:…) with no flag twice and at least one; lookbehinds never quantified.
  • \p{…} and \P{…}: General_Category, Script and Script_Extensions values and binary properties by exact name, from src/regexp/names.rs. scripts/gen-regexp-names.py writes it from Unicode 17.0's PropertyValueAliases.txt, the version src/unicode.rs and test262 use, and ECMA-262's tables of binary properties and properties of strings; the binary list matches crates/parsec's.
  • Each error has a stable code, tabled in the module docs: regexp-flags, regexp-nothing-to-repeat, regexp-quantifier-range, regexp-lone-bracket, regexp-unterminated, regexp-group, regexp-modifiers, regexp-group-name, regexp-duplicate-group-name, regexp-unknown-group-name, regexp-backreference, regexp-escape, regexp-range-order, regexp-range-class, regexp-property, regexp-negated-strings, regexp-class-syntax, too-deep. Spans are byte offsets in the pattern.
  • Limits: groups and v classes nest to DEFAULT_MAX_DEPTH, then too-deep; only three small functions sit on the stack per level, so 256 levels fit a 1 MB thread in a debug build. Duplicate group names are counted per alternative and disjunction, so 100,000 alternatives of one name check in linear time, not quadratic.
  • tests/test262.rs validates every regular expression literal's pattern. The path fragments for patterns leave early_error; the two undeclared-export ones stay for the next PR.
  • A regexp fuzz target (each input as a pattern with no flag, u, v and gimsyd) joins the nightly matrix in fuzz.yml, and the parse target validates every literal it parses. CONTRIBUTING lists the new target.
  • README gets a row, ROADMAP's M11 a bullet, and the changelog an entry.

Tests. tests/regexp.rs, 15 tests, each with sources that must pass and sources that must fail with a given code, in each mode the rule touches:

  • flags, atoms_and_alternatives, quantifiers (including B.1.2's quantified lookahead and lone braces), brackets_need_escaping_in_unicode_mode, groups, modifiers, group_names (escaped and astral names, duplicates in and out of separate alternatives), backreferences (forward ones, and B.1.2's octal fallback), character_escapes, classes (ranges, B.1.2's [\d-a] and [\c1], the surrogate-pair range), property_escapes (39 names and values, right and wrong, and properties of strings), v_flag_classes (15 valid set expressions and 43 invalid ones, among them one for each reserved double punctuator).
  • spans_point_at_the_trouble, with multi-byte text before the error.
  • deep_nesting_is_an_error_not_a_crash: 100,000-deep groups and v classes are too-deep on a 1 MB thread; 200 deep is fine.
  • wide_patterns_take_linear_time: 100,000 same-name groups, classes and backreferences.

test262 goes from 4,014 to 4,372 of the 4,374 parse negatives caught: all 358 pattern tests now fail through validate, with 0 failures. Every regular expression in the 48,408 files that should parse validates, including built-ins/RegExp/property-escapes' 442 generated files, which use every property value; that's the check on false positives.

cargo +nightly fuzz run regexp -- -max_total_time=240 -max_len=256: 4,852,130 runs, no crash. cargo +nightly fuzz run parse -- -max_total_time=150 -max_len=4096: 1,510,068 runs, no crash. cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and the bench's, cargo fmt --check for both, cargo doc for each crate with -D warnings, cargo deny check licenses bans sources for both, the fuzz manifests' cargo metadata and scripts/check-headers.sh pass locally.

The lexer finds where a regular expression literal ends and checks its
flags, but nothing read the pattern, so /[z-a]/, /(?<a>x)(?<a>y)/ and
/\p{Lettr}/u parsed, where an engine throws a SyntaxError before
running anything. 358 of test262's parse negatives were skipped for
it.

parsec_js::regexp::validate reads a pattern by ECMA-262's pattern
grammar and its early errors, in the three modes the flags choose:
without u, with Annex B's B.1.2 (legacy octal and identity escapes,
\c, a lookahead quantified, braces as characters, [\d-a] as a union,
code units rather than code points); with u; and with v's nested
classes, && and --, \q{...} and properties of strings, with what may
match a string kept from being negated. Named groups, ES2025's
duplicate names in separate alternatives, (?ims-ims:...) modifiers,
backreferences, quantifier ranges and every \p{...} name and value in
Unicode 17.0 are checked. Each error has a stable code, listed in the
module docs, and a span in the pattern.

The names come from scripts/gen-regexp-names.py, which reads the UCD's
PropertyValueAliases.txt and copies ECMA-262's tables of binary
properties. Nesting stops at DEFAULT_MAX_DEPTH with too-deep, and fits
a 1 MB thread in a debug build; duplicate group names are counted per
alternative, so checking them is linear.

test262 now validates every literal's pattern: all 358 negatives fail
and every file that should parse still does. A regexp fuzz target
joins the nightly run, and the parse target validates each literal.
Brings the branch onto its new base after the pull request below it merged.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

javascript The JavaScript parser, parsec-js

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant