Skip to content

deps: update simdutf to 9.1.0 and transcode UTF-8 writes in a single pass - #65390

Closed
lemire wants to merge 2 commits into
nodejs:mainfrom
lemire:deps-simdutf-9.1.0
Closed

deps: update simdutf to 9.1.0 and transcode UTF-8 writes in a single pass#65390
lemire wants to merge 2 commits into
nodejs:mainfrom
lemire:deps-simdutf-9.1.0

Conversation

@lemire

@lemire lemire commented Aug 18, 2026

Copy link
Copy Markdown
Member

Follow-up to #65324.

Update deps/v8/third_party/simdutf from 7.7.0 to 9.1.0 (upstream
2d4e249d, single-header release). StringBytes::Write() then uses
convert_utf16_to_utf8_with_replacement() so two-byte UTF-8 writes no
longer validate and copy through to_well_formed_utf16(). Unpaired
surrogates still become U+FFFD. Truncating writes and strings of 32
code units or fewer still go through V8.

benchmark/buffers/buffer-write-string-utf8.js on Linux x64 (Xeon Gold
6548N, 20 runs, significance as in compare.R):

len=65536 chars='two-byte-astral'          ***  +46.14%  ±0.14%
len=65536 chars='two-byte-lone-surrogate'  ***  +31.33%  ±0.13%
len=2048  chars='two-byte-astral'          ***  +39.85%  ±0.14%
len=2048  chars='two-byte-lone-surrogate'  ***  +19.99%  ±0.34%
len=65536 chars='two-byte'                 ***   +9.79%  ±0.12%
len=2048  chars='two-byte'                 ***   +4.41%  ±0.24%
one-byte strings / length <= 32                    ~0%

Refs: https://github.com/simdutf/simdutf/releases/tag/v9.1.0
Refs: #65324

lemire added 2 commits August 18, 2026 13:30
Refresh deps/v8/third_party/simdutf from the v9.1.0 single-header
release (upstream 2d4e249d).

Refs: https://github.com/simdutf/simdutf/releases/tag/v9.1.0
Signed-off-by: Daniel Lemire <daniel@lemire.me>
StringBytes::Write() validated UTF-16 and, on unpaired surrogates,
copied through to_well_formed_utf16() before converting. simdutf
9.1.0 can replace unpaired surrogates during the convert, so drop
the extra pass.

Signed-off-by: Daniel Lemire <daniel@lemire.me>
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/security-wg
  • @nodejs/v8-update

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Aug 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Benchmark GHA (buffers / buffer-write-string-utf8): https://github.com/nodejs/node/actions/runs/32170665721

@lemire
lemire requested a review from codebytere August 18, 2026 18:42
@lemire lemire changed the title deps: update simdutf to 9.1.0 deps: update simdutf to 9.1.0 and transcode UTF-8 writes in a single pass Aug 18, 2026
@lemire

lemire commented Aug 18, 2026

Copy link
Copy Markdown
Member Author

@anonrig was first to push this feature to simdutf (convert_utf16_to_utf8_with_replacement), but it took some time to get it published and fast. By using the fast convert_utf16_to_utf8_with_replacement in simdutf 9.1, we can simplify the recent change by @codebytere while improving the performance in some cases.

@lemire
lemire requested review from anonrig and jasnell August 18, 2026 18:46
@lemire lemire added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 18, 2026

@anonrig anonrig left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately we need to wait for v8 to update its simdutf version or use our own simdutf as a dependency and make sure v8 uses our own simdutf

@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 18, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codecov

codecov Bot commented Aug 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.19%. Comparing base (55e4ca3) to head (3da3299).

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #65390      +/-   ##
==========================================
+ Coverage   90.14%   90.19%   +0.05%     
==========================================
  Files         752      752              
  Lines      251870   251866       -4     
  Branches    47365    47407      +42     
==========================================
+ Hits       227037   227180     +143     
+ Misses      16177    16031     -146     
+ Partials     8656     8655       -1     
Files with missing lines Coverage Δ
src/string_bytes.cc 75.00% <100.00%> (-0.24%) ⬇️

... and 43 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@lemire

lemire commented Aug 18, 2026

Copy link
Copy Markdown
Member Author

@anonrig Chromium is at 7.7.0. Let us see if we can't get them to update.

@lemire lemire closed this Aug 18, 2026
@lemire

lemire commented Aug 18, 2026

Copy link
Copy Markdown
Member Author

I'm closing this, but let us remember to come back to it.

For now, what @codebytere did works fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants