Skip to content

CSS API: Skip UTF-8 decoding of ASCII bytes after an identifier - #106

Open
sirreal wants to merge 2 commits into
css-api/css-token-processor-foundationsfrom
css-api/tokenizer-ascii-ident-start
Open

sirreal wants to merge 2 commits into
css-api/css-token-processor-foundationsfrom
css-api/tokenizer-ascii-ident-start

Conversation

@sirreal

@sirreal sirreal commented Nov 17, 2026

Copy link
Copy Markdown
Owner

Caution

This is a work in progress draft and is not ready for any serious review.

Return early from consume_ident_start_codepoint() for ASCII bytes that cannot start an identifier. The method runs after every identifier on the byte that ends it, usually :, ,, ;, ( or a space. For those bytes it ran _wp_scan_utf8() and utf8_ord() before returning 0. An ASCII byte other than a letter, _ or NUL never starts an identifier, so the method now returns 0 before it decodes.

No behavior changes. The css-api group passes (1007 tests).

Tokenizing 257 KB of CSS (buttons.css, admin-bar.css, dashicons.css and Twenty Twenty-One's style.css, 51,336 tokens) takes 15.2 ms instead of 18.9 ms: best of 15 runs, one process per variant, three rounds, PHP 8.5.

Depends on WordPress#14781.

Trac ticket: TBD

Use of AI Tools

AI assistance: Yes
Tools: Claude Code
Models: Claude Opus 5.5
Used for: implementation, measurement and this description.


This pull request is for code review only. Keep other discussion in the Trac ticket. Do not merge. See GitHub Pull Requests for Code Review in the Core Handbook.

consume_ident_sequence() scans ASCII name bytes with strspn() and then
asks consume_ident_start_codepoint() about the byte that stopped the
scan. For an ASCII byte such as `,` `:` or a space, that method ran
_wp_scan_utf8() and utf8_ord() before returning 0. An ASCII byte other
than a letter, `_` or NUL never starts an identifier, so the method
returns 0 for it before decoding.

Tokenizing a 70 KB stylesheet (a generated global stylesheet with the
admin-bar and buttons styles, 15,979 tokens) took 6.2 ms before and
4.9 ms after, best of 10 runs on PHP 8.5.

(cherry picked from commit 0dab3d9)
@github-actions

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs.

Core Committers: Use this line as a base for the props when committing in SVN:

Props jonsurrell.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.


Props Bot commands

You can trigger Props Bot actions by commenting on this PR:

  • @propsbot add <username> to give props to a GitHub or WordPress.org user. Requires write access.
  • @propsbot remove <username> to leave a user out of the props. Requires write access.
  • @propsbot refresh to refresh this list. Adding the props-bot label works too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant