deps: update simdutf to 9.1.0 and transcode UTF-8 writes in a single pass - #65390
deps: update simdutf to 9.1.0 and transcode UTF-8 writes in a single pass#65390lemire wants to merge 2 commits into
Conversation
Refresh deps/v8/third_party/simdutf from the v9.1.0 single-header release (upstream 2d4e249d). Refs: https://github.com/simdutf/simdutf/releases/tag/v9.1.0 Signed-off-by: Daniel Lemire <daniel@lemire.me>
StringBytes::Write() validated UTF-16 and, on unpaired surrogates, copied through to_well_formed_utf16() before converting. simdutf 9.1.0 can replace unpaired surrogates during the convert, so drop the extra pass. Signed-off-by: Daniel Lemire <daniel@lemire.me>
|
Review requested:
|
|
Benchmark GHA (buffers / buffer-write-string-utf8): https://github.com/nodejs/node/actions/runs/32170665721 |
|
@anonrig was first to push this feature to simdutf ( |
anonrig
left a comment
There was a problem hiding this comment.
Unfortunately we need to wait for v8 to update its simdutf version or use our own simdutf as a dependency and make sure v8 uses our own simdutf
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #65390 +/- ##
==========================================
+ Coverage 90.14% 90.19% +0.05%
==========================================
Files 752 752
Lines 251870 251866 -4
Branches 47365 47407 +42
==========================================
+ Hits 227037 227180 +143
+ Misses 16177 16031 -146
+ Partials 8656 8655 -1
🚀 New features to boost your workflow:
|
|
@anonrig Chromium is at 7.7.0. Let us see if we can't get them to update. |
|
I'm closing this, but let us remember to come back to it. For now, what @codebytere did works fine. |
Follow-up to #65324.
Update
deps/v8/third_party/simdutffrom 7.7.0 to 9.1.0 (upstream2d4e249d, single-header release).
StringBytes::Write()then usesconvert_utf16_to_utf8_with_replacement()so two-byte UTF-8 writes nolonger validate and copy through
to_well_formed_utf16(). Unpairedsurrogates still become U+FFFD. Truncating writes and strings of 32
code units or fewer still go through V8.
benchmark/buffers/buffer-write-string-utf8.json Linux x64 (Xeon Gold6548N, 20 runs, significance as in
compare.R):Refs: https://github.com/simdutf/simdutf/releases/tag/v9.1.0
Refs: #65324