buffer: support aligned allocations - #65003
Conversation
|
Review requested:
|
Add an optional `alignment` argument to `Buffer.allocUnsafe()` and
`Buffer.allocUnsafeSlow()`, which guarantees that the memory backing the
returned buffer starts at an address that is a multiple of `alignment`.
Some operating system interfaces refuse to work with unaligned memory.
The motivating case is unbuffered ("direct") file I/O: a read or write
on a descriptor opened with `O_DIRECT` fails with `EINVAL` unless the
buffer address is a multiple of the logical block size of the underlying
device. Until now there was no way to obtain such a buffer from JS,
since the address of a backing store can neither be observed nor chosen.
Alignment is also worth having purely for performance, for instance to
keep a hot buffer from straddling one more cache line than its size
requires.
V8 does not allow picking the address of a backing store, so alignment
is instead achieved by over-allocating `alignment - 1` bytes and
positioning the buffer at the first suitably aligned byte within them.
The new `arrayBufferAlignedOffset()` binding computes that offset.
The `Buffer.allocUnsafe()` pool is now aligned to a cache line itself,
which lets pooled allocations satisfy any alignment up to 64 bytes by
padding their offset into the pool rather than allocating separately.
Assisted-by: Claude/Opus 5
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1e1a826 to
ef04e22
Compare
There was a problem hiding this comment.
Pull request overview
This PR adds optional alignment support to Buffer.allocUnsafe() and Buffer.allocUnsafeSlow() so callers can request that the returned Buffer’s backing memory starts at an address aligned to a power-of-two boundary, enabling use cases like O_DIRECT I/O and some alignment-based performance optimizations.
Changes:
- Add a new internal binding (
arrayBufferAlignedOffset) and supporting C++ allocation helper to compute aligned view offsets into over-allocated ArrayBuffers. - Implement aligned allocation plumbing in
lib/buffer.js(including cache-line-aligned pool base and aligned pooled slicing up to 64 bytes). - Add documentation and tests covering aligned allocation behavior and argument validation.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| test/parallel/test-buffer-alloc-alignment.js | Adds test coverage for aligned allocations and validation behavior. |
| src/node_buffer.cc | Introduces AllocateUnsafeArrayBuffer() and the arrayBufferAlignedOffset() binding. |
| lib/internal/buffer.js | Adds createUnsafeAlignedBuffer() used by Buffer APIs and pool creation. |
| lib/buffer.js | Adds alignment parameter handling, alignment validation, and aligns the pool base. |
| doc/api/worker_threads.md | Updates Buffer.allocUnsafe() documentation link anchor. |
| doc/api/deprecations.md | Updates Buffer.allocUnsafe/allocUnsafeSlow documentation link anchors. |
| doc/api/buffer.md | Documents the new alignment argument and adds an “Aligned allocations” section. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const buf = alloc(size, alignment); | ||
| assert.strictEqual(buf.length, size); | ||
| assert.strictEqual(buf.byteOffset % alignment, 0); | ||
| // The view must fit inside the (over-allocated) ArrayBuffer. |
| * @param {number} size | ||
| * @param {number} [alignment] A power of two, at most 2 ** 30 | ||
| * @returns {FastBuffer|undefined} | ||
| */ | ||
| Buffer.allocUnsafeSlow = function allocUnsafeSlow(size) { | ||
| Buffer.allocUnsafeSlow = function allocUnsafeSlow(size, alignment) { |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #65003 +/- ##
========================================
Coverage 90.27% 90.27%
========================================
Files 762 762
Lines 247534 247634 +100
Branches 46694 46715 +21
========================================
+ Hits 223457 223553 +96
+ Misses 15529 15498 -31
- Partials 8548 8583 +35
🚀 New features to boost your workflow:
|
Adds an optional
alignmentargument toBuffer.allocUnsafe()andBuffer.allocUnsafeSlow(), guaranteeing that the memory backing the returnedbuffer starts at an address that is a multiple of
alignment.Why
Some operating system interfaces refuse to work with unaligned memory. The
motivating case is unbuffered ("direct") file I/O: a read or write on a
descriptor opened with
O_DIRECTfails withEINVALunless the buffer addressis a multiple of the logical block size of the underlying device. There is
currently no way to obtain such a buffer from JS, since the address of a backing
store can neither be observed nor chosen.
Alignment is also worth having purely for performance, for instance to keep a hot
buffer from straddling one more cache line than its size requires.
How
V8 does not allow picking the address of a backing store, so alignment is
achieved by over-allocating
alignment - 1bytes and positioning the buffer atthe first suitably aligned byte within them. A new
arrayBufferAlignedOffset()binding computes that offset.
It returns an offset rather than a finished
Bufferfor two reasons: externalbacking stores are rejected outright when the V8 sandbox is enabled, and
Buffer::New()is not usable duringbuffer.jsevaluation — the bufferprototype is only wired up after
require('buffer')returns, which the poolcreation runs before.
Consequently
buf.byteOffsetis usually non-zero andbuf.bufferis larger thansize, as is already the case for pooled buffers. This is documented.The
Buffer.allocUnsafe()pool is now cache-line aligned itself, so pooledallocations can satisfy any alignment up to 64 bytes by padding their offset into
the pool rather than allocating separately.
poolOffsetis therefore trackedrelative to a new
poolBase.alignmentmust be a power of two no larger than2 ** 30, andsize + alignment - 1must fit withinbuffer.constants.MAX_LENGTH.Notes for reviewers
binding
CHECKs it; the added test sweeps sizes × alignments over both APIs toexercise those assertions.