Skip to content
NCodeGroupPublic

About

Pinned, pooled .NET buffers that zero themselves on dispose, for keys and other secrets, plus fixed-size buffer writers and string splitting without substring copies.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

735 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

NCode

NCode.Buffers

Pinned, pooled .NET buffers that zero themselves on dispose, for keys and other secrets, plus fixed-size buffer writers and string splitting without substring copies.

NuGet version NuGet downloads CI Coverage Target framework License OpenSSF Scorecard OpenSSF Best Practices

The documentation site has a quick start, a guide for each task, the concepts behind the buffers, the evidence for each guarantee and a reference for every public member.

Installation

dotnet add package NCode.Buffers

The target frameworks are the ones the package's .csproj declares; they are listed on nuget.org's Frameworks tab and shown by the badge above. CHANGELOG.md records when a framework is added or dropped and which release line to stay on. The package depends on Nerdbank.Streams, whose Sequence<T> the pooled buffer writers return.

Quick start

This example derives a key from a password, verifies an HMAC-SHA256 signature with it, and zeroes the derived key and the expected MAC when the method returns.

using System.Security.Cryptography;
using NCode.Buffers;

public static class Signatures
{
    public static bool VerifySignature(
        ReadOnlySpan<byte> password,
        ReadOnlySpan<byte> salt,
        ReadOnlySpan<byte> payload,
        ReadOnlySpan<byte> signature
    )
    {
        // A pinned, pooled 32-byte key, zeroed when keyOwner is disposed.
        using var keyOwner = BufferFactory.Rent(32, isSensitive: true, out Span<byte> key);
        Rfc2898DeriveBytes.Pbkdf2(
            password,
            salt,
            key,
            iterations: 600_000,
            HashAlgorithmName.SHA256
        );

        // The expected MAC lives on the stack and is zeroed when expectedLifetime is disposed.
        Span<byte> expected = stackalloc byte[HMACSHA256.HashSizeInBytes];
        using var expectedLifetime = expected.GetSecureLifetime();
        HMACSHA256.HashData(key, payload, expected);

        // Constant time, so the comparison doesn't reveal where the MACs differ.
        return CryptographicOperations.FixedTimeEquals(expected, signature);
    }
}

See Choosing a type for the other buffers.

Choosing a type

Pick the row that matches your data and where the buffer lives. Rent and the Create… methods are on BufferFactory, GetSecureLifetime is an extension method on Span<T>, and the other rows are types.

For secrets

Each of these zeroes its memory on dispose, and all but GetSecureLifetime, which wraps memory you already have, pin it so the GC never copies it. Rent, CreatePooledBufferWriter, RentRandom, RentBytes, RentChars, RentDecoded, RentEncoded, the growable buffers and PooledTextBuilder take isSensitive: true; SecureMemoryPool<T>, CreatePinnedArray, GetSecureLifetime and Secret.Create take no such argument, because they always give you the secure kind.

API Use it when Pooled
Rent You need a temporary buffer for a key, password or other secret Yes¹
CreatePooledBufferWriter You write a secret whose length you don't know up front Yes¹
RentRandom, RentBytes, RentDecoded You make a key from random bytes, text, Base64 or hex Yes¹
CreateContiguousBufferWriter You write a secret of unknown length that must end up in one span Yes¹
CreatePooledStream An API writes or reads the secret through a Stream Yes¹
PooledTextBuilder You build secret text from parts Yes¹
Secret<T> You keep a secret in a field without it reaching a log Yes¹
SecureMemoryPool<T> An API takes a MemoryPool<T> Yes¹
CreatePinnedArray An API needs a T[], within one synchronous scope No
GetSecureLifetime You already have the memory, usually stackalloc No

¹ Up to 4096 bytes, SecureMemoryPool<T>.PageSize. A larger buffer gets its own pinned array, still zeroed on dispose.

For other data

Nothing here is pinned or zeroed.

API Use it when Pooled
Rent, CreatePooledBufferWriter You want pooled buffers: pass isSensitive: false Yes
CreateContiguousBufferWriter, CreatePooledStream, PooledTextBuilder You want a pooled buffer that grows: pass isSensitive: false Yes
CreateArrayBufferWriter You want a simple growable writer No
FixedSpanBufferWriter<T> You write into a fixed-size span, such as stackalloc, in synchronous code No
FixedMemoryBufferWriter<T> You write into fixed-size Memory<T> and keep the writer in a field or across await No

Features

Secure memory

BufferFactory

The entry point. isSensitive: true takes buffers from SecureMemoryPool<T> (pinned, zeroed on dispose), while isSensitive: false takes them from MemoryPool<T>.Shared (neither). Rent hands back a span sliced to exactly the requested length.

// A pooled buffer that is pinned and zeroed on dispose, sliced to exactly 256 bytes.
using var owner = BufferFactory.Rent(256, isSensitive: true, out Span<byte> buffer);

// A pinned array, for an API that needs a byte[].
using var lifetime = BufferFactory.CreatePinnedArray(128);
byte[] array = lifetime;

// A writer for a secret of unknown length, built from pinned, pooled segments.
using var writer = BufferFactory.CreatePooledBufferWriter(isSensitive: true);
"payload"u8.CopyTo(writer.GetSpan(7));
writer.Advance(7);

// A growable writer for data that isn't secret.
ArrayBufferWriter<byte> arrayWriter = BufferFactory.CreateArrayBufferWriter(1024);

SecureMemoryPool<T>

A MemoryPool<T> whose buffers are pinned and zeroed with CryptographicOperations.ZeroMemory when returned. After a garbage collection, on the finalizer thread, it drops its cached buffers when the memory load is at or above HighPressureThreshold (default 0.9) times the GC's high memory load threshold. That check follows every generation 2 collection, and the first one or two after the pool is created can follow a generation 0 or 1 collection.

SecureMemoryPool<T>.Shared serves the whole process, so its Dispose does nothing; dispose only a pool you create.

using IMemoryOwner<byte> owner = SecureMemoryPool<byte>.Shared.Rent(1024);
Memory<byte> memory = owner.Memory; // at least 1024 bytes: slice it to the length you need

SecureArrayLifetime<T> and SecureSpanLifetime<T>

ref structs that zero their memory on Dispose. SecureArrayLifetime<T> allocates a pinned array, while SecureSpanLifetime<T> wraps memory you already have and doesn't pin it.

// A pinned array, zeroed when lifetime is disposed.
using var lifetime = SecureArrayLifetime<byte>.Create(256);
Span<byte> buffer = lifetime;

// Memory you already have, zeroed when secure is disposed.
Span<byte> stackBuffer = stackalloc byte[128];
using var secure = stackBuffer.GetSecureLifetime();

Random bytes

BufferFactory.RentRandom fills a rented buffer from RandomNumberGenerator.

// 32 random bytes in a pinned, pooled buffer that is zeroed on dispose.
using var owner = BufferFactory.RentRandom(32, isSensitive: true, out Span<byte> key);

Buffer writers

FixedSpanBufferWriter<T>

A ref struct IBufferWriter<T> over a span you supply. GetSpan or Advance past its capacity throws InvalidOperationException.

Span<byte> buffer = stackalloc byte[256];
var writer = new FixedSpanBufferWriter<byte>(buffer);

"abcd"u8.CopyTo(writer.GetSpan(4));
writer.Advance(4);
ReadOnlySpan<byte> written = writer.WrittenSpan;

FixedMemoryBufferWriter<T>

The same over Memory<T>, as a class you can keep in a field or use across await.

var writer = new FixedMemoryBufferWriter<byte>(new byte[256]);
using (var json = new Utf8JsonWriter(writer))
{
    JsonSerializer.Serialize(json, new { Id = 42, Name = "example" });
}
await stream.WriteAsync(writer.WrittenMemory);

Growable buffers

ContiguousBufferWriter<T>

An IBufferWriter<T> from BufferFactory.CreateContiguousBufferWriter that keeps its data in one span. With isSensitive: true, it zeroes each buffer it outgrows.

using var writer = BufferFactory.CreateContiguousBufferWriter(isSensitive: true);
using (var json = new Utf8JsonWriter(writer))
{
    JsonSerializer.Serialize(json, new { Id = 42 });
}

// The written data is always one span, so it can be hashed without a copy.
byte[] hash = SHA256.HashData(writer.WrittenSpan);

PooledMemoryStream

A seekable, read-write Stream over one pooled buffer, from BufferFactory.CreatePooledStream. With isSensitive: true, it zeroes the buffers it outgrows and the bytes SetLength cuts off.

using var stream = BufferFactory.CreatePooledStream(isSensitive: true);
stream.Write("payload"u8);
stream.Position = 0;

Span<byte> buffer = stackalloc byte[7];
int read = stream.Read(buffer);

PooledTextBuilder

A ref struct text builder that can start in a stackalloc span. It can't cross an await, you must never copy it, and ToString() makes a string that can't be zeroed.

// Starts in a stack buffer, and moves to a pooled one only if the text outgrows it.
Span<char> start = stackalloc char[64];
using var builder = new PooledTextBuilder(start, isSensitive: true);
builder.Append("token-");
builder.Append(42);
ReadOnlySpan<char> text = builder.WrittenSpan;

Sequences

MemorySegment<T>

Chains memory blocks into a ReadOnlySequence<T>.

var first = new MemorySegment<byte>((byte[])[1, 2, 3]);
var last = first.Append((byte[])[4, 5, 6]);
var sequence = new ReadOnlySequence<byte>(first, 0, last, last.Memory.Length);
// sequence holds 1, 2, 3, 4, 5, 6

SequenceExtensions and RefSpanLease<T>

GetSpanLease returns a RefSpanLease<T>, a span paired with the owner to dispose. Disposing the lease leaves the writer valid when its data is in one segment; when it spans several segments, it only releases the rented copy.

using var writer = BufferFactory.CreatePooledBufferWriter(isSensitive: true);
writer.Write("payload"u8);

// No copy when the data is in one segment; otherwise it is copied into a rented buffer.
using (var lease = writer.GetSpanLease(isSensitive: true))
{
    Process(lease.Span);
}

ConsumeAsContiguousSpan takes ownership of the data and returns its owner.

using var writer = BufferFactory.CreatePooledBufferWriter(isSensitive: true);
writer.Write("payload"u8);

// owner now owns the data; don't use writer again.
using var owner = writer.ConsumeAsContiguousSpan(
    isSensitive: true,
    out ReadOnlySpan<byte> span
);
Process(span);

ReadOnlySequence<T> leases

GetSpanLease also works on a ReadOnlySequence<T>. It reads the sequence as one span and copies only when the sequence spans several segments.

byte[] head = "pay"u8.ToArray();
byte[] tail = "load"u8.ToArray();
var first = new MemorySegment<byte>(head);
var last = first.Append(tail);
var sequence = new ReadOnlySequence<byte>(first, 0, last, last.Memory.Length);

// The data spans two segments, so it is copied into a rented buffer.
using var lease = sequence.GetSpanLease(isSensitive: true);
Process(lease.Span);

EmptyMemory<T>

A shared, empty IMemoryOwner<T> whose Dispose does nothing.

IMemoryOwner<byte> empty = EmptyMemory<byte>.Singleton;

String splitting

StringSegments.Split and the SplitSegments extension split a string or ReadOnlyMemory<char> on a char or a string separator, optionally with a StringComparison, without copying substrings: each segment's Memory is a slice of the original, and the result is an IReadOnlyCollection<ReadOnlySequenceSegment<char>>. It isn't zero-allocation: it allocates one small segment object per piece. A separator that matches no text, such as an empty one, returns one segment that holds the whole value, as string.Split(string) does.

foreach (var segment in "one,two,three".SplitSegments(','))
{
    Console.WriteLine(segment.Memory.ToString());
}

StringSegments segments = StringSegments.Split(
    "aSEPbSEPc",
    "sep",
    StringComparison.OrdinalIgnoreCase
);
int count = segments.Count; // 3

Secure encoding

SecureEncoding.UTF8 and SecureEncoding.ASCII throw DecoderFallbackException or EncoderFallbackException on invalid input instead of substituting U+FFFD or ?. SecureEncoding.UTF8 writes no byte order mark.

byte[] bytes = SecureEncoding.UTF8.GetBytes("Hello, World!");
string text = SecureEncoding.UTF8.GetString(bytes);

// Invalid UTF-8 throws instead of decoding to U+FFFD, so untrusted input can be rejected.
bool isValid;
try
{
    SecureEncoding.UTF8.GetString((byte[])[0xC3, 0x28]);
    isValid = true;
}
catch (DecoderFallbackException)
{
    isValid = false;
}

Text to and from a rented buffer

RentBytes and RentChars on an Encoding encode or decode into a rented buffer of the exact length, so a secret never passes through a string or an unpooled array.

// Encode text to UTF-8 bytes in a rented buffer, and decode the bytes back to chars.
using var bytesOwner = SecureEncoding.UTF8.RentBytes(
    "secret",
    isSensitive: true,
    out Span<byte> utf8
);
using var charsOwner = SecureEncoding.UTF8.RentChars(
    utf8,
    isSensitive: true,
    out Span<char> chars
);

Base64, Base64url and hex

BinaryTextEncoding.RentDecoded and RentEncoded convert between bytes and Base64, Base64url, or upper or lower case hex text, with BinaryTextFormat choosing the format. Base64UrlStrict decodes only the canonical unpadded Base64url JOSE requires, and refuses = padding, whitespace and unused bits set in the last character. Invalid text throws FormatException after the rental is returned.

// Decode Base64Url text to bytes, then encode the bytes as lowercase hex.
using var bytesOwner = BinaryTextEncoding.RentDecoded(
    "-_8B",
    BinaryTextFormat.Base64Url,
    isSensitive: true,
    out Span<byte> bytes
);
using var textOwner = BinaryTextEncoding.RentEncoded(
    bytes,
    BinaryTextFormat.HexLower,
    isSensitive: true,
    out Span<char> hex
);

Secret holder

Secret<T>

Secret.Create copies a secret into a pinned, pooled buffer that Dispose zeroes, and returns a Secret<T> that you keep in a field. ToString() and the debugger show [REDACTED], so the value doesn't reach a log; Reveal() reads it. Equals compares in fixed time.

using var secret = Secret.Create("hunter2"u8);
ReadOnlySpan<byte> value = secret.Reveal();
string text = secret.ToString(); // [REDACTED]

Metrics

SecureMemoryPool<T> reports its rentals, returns and trimmed buffers through a System.Diagnostics.Metrics meter named by BufferMetrics.MeterName. docs/metrics.md lists each instrument. Subscribe to the meter by that name, with OpenTelemetry's AddMeter(BufferMetrics.MeterName) or a MeterListener:

var totals = new ConcurrentDictionary<string, long>();

using var listener = new MeterListener();
listener.InstrumentPublished = (instrument, l) =>
{
    if (instrument.Meter.Name == BufferMetrics.MeterName)
    {
        l.EnableMeasurementEvents(instrument);
    }
};
listener.SetMeasurementEventCallback<long>(
    (instrument, value, tags, state) =>
        totals.AddOrUpdate(instrument.Name, value, (_, total) => total + value)
);
listener.Start();

CryptographicOperations extensions

ZeroMemory<T> and FixedTimeEquals<T> work on spans of any unmanaged type, a value type without reference fields at any depth; a T with references doesn't compile. These are C# 14 extension members on CryptographicOperations, which a project compiles with by default on any framework this package targets; a project that sets an older LangVersion calls them on CryptographicOperationsExtensions instead, as the sample's last line does.

Span<int> sensitiveData = stackalloc int[4];
// ... use the sensitive data ...
CryptographicOperations.ZeroMemory(sensitiveData);

ReadOnlySpan<ulong> computed = ComputeMac(data);
ReadOnlySpan<ulong> expected = GetExpectedMac();
bool isValid = CryptographicOperations.FixedTimeEquals(computed, expected);

// The same methods, callable from any C# version:
CryptographicOperationsExtensions.ZeroMemory(sensitiveData);

Security notes

Pinning

  • Does: sensitive buffers come from GC.AllocateUninitializedArray<T>(length, pinned: true) on the Pinned Object Heap, so the GC never moves them and leaves no stale copy at an old address, and native code can rely on the address.
  • Does not: lock the memory into RAM. The OS can still write it to a page file, swap or hibernation file.
  • Does not: hide the memory from a debugger, a crash dump, or another process that can read this one's memory.
  • SecureSpanLifetime<T> doesn't pin.

Zeroing

  • Does: Dispose zeroes the whole underlying buffer with CryptographicOperations.ZeroMemory, which the JIT doesn't remove. That includes the part beyond the slice Rent returned.
  • Does: zero a pooled buffer before another caller can rent it.
  • Does: with isSensitive: true, the growable writer, stream and text builder zero each buffer they outgrow.
  • Does not: run without Dispose. There is no finalizer, so always use using.
  • Does not: clear copies, such as a string, a ToArray() result, or a buffer rented with isSensitive: false.
  • A newly allocated buffer is not zeroed when rented, so write before you read.
  • Using a span after its owner is disposed is a bug. For a pooled buffer, from Rent, CreatePooledBufferWriter or SecureMemoryPool<T>, the span can read and write memory the pool has since rented to another caller. A SecureArrayLifetime<T>'s array and the memory under a SecureSpanLifetime<T> belong to no pool: the span still reaches the same memory, which Dispose zeroed.

Comparing and encoding

  • FixedTimeEquals takes the same time for any contents of equal length, but returns false at once when the lengths differ, so it doesn't hide the length.
  • A string decoded with SecureEncoding can't be zeroed. Decode a secret into a sensitive buffer with GetChars(ReadOnlySpan<byte>, Span<char>) instead.
  • Secret<T>.ToString() and the debugger show [REDACTED]; Equals compares in fixed time and reveals only the length.

References

Project

About

Pinned, pooled .NET buffers that zero themselves on dispose, for keys and other secrets, plus fixed-size buffer writers and string splitting without substring copies.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages