Skip to content

Allocate AWSCredentials secret key and signing key through a dedicated memory system - #39

Open
mstetsyuk wants to merge 1 commit into
ClickHouse/1.11.771from
sensitive-string-allocator
Open

Allocate AWSCredentials secret key and signing key through a dedicated memory system#39
mstetsyuk wants to merge 1 commit into
ClickHouse/1.11.771from
sensitive-string-allocator

Conversation

@mstetsyuk

Copy link
Copy Markdown
Member

Issue #, if available: ClickHouse-side change: ClickHouse/ClickHouse#82441 (revival; the new pull request will link back here)

Description of changes:

Lets the application keep the AWS secret access key out of core dumps by allocating it through a dedicated memory system.

  • MemoryManagementOptions::sensitiveMemoryManager: a second MemorySystemInterface that InitAPI installs (and ShutdownAPI removes) regardless of USE_AWS_MEMORY_MANAGEMENT. Defaults to nullptr, in which case nothing changes.
  • Aws::SensitiveAllocator<T> and Aws::SensitiveString: a minimal allocator (not derived from std::allocator, so allocate_at_least cannot bypass it) and the string type built on it. They fall back to malloc/free when no sensitive memory manager is installed.
  • AWSCredentials::m_secretKey is an Aws::SensitiveString; GetAWSSecretKey returns it. A constructor taking const Aws::SensitiveString& lets callers hand over a secret without a plain Aws::String copy.
  • AWSAuthV4Signer and AWSAuthEventStreamV4Signer keep the derived signing key and m_currentSecretKey in Aws::SensitiveString.
  • HashingUtils::CalculateSHA256HMAC(const Aws::String&, const Aws::SensitiveString&): computes the first HMAC step ("AWS4" + secret) without materialising the concatenation in a plain string.
  • The three smithy identity resolvers are adjusted for the new GetAWSSecretKey return type.

ClickHouse installs a MADV_DONTDUMP jemalloc-arena allocator into sensitiveMemoryManager, so every AWSCredentials held by a credentials provider, and every signing key, lives in pages excluded from core dumps.

Check all that applies:

  • Did a review by yourself.
  • Added proper tests to cover this PR. (The behaviour is verified on the ClickHouse side with a /proc/self/smaps VmFlags: dd check on GetAWSSecretKey().data() after installing the manager; the SDK itself only changes types.)
  • Checked if this PR is a breaking (APIs have been changed) change. GetAWSSecretKey returns const Aws::SensitiveString& instead of const Aws::String&; callers that need an Aws::String construct one from .data()/.size().
  • Checked if this PR will not introduce cross-platform inconsistent behavior.
  • Checked if this PR would require a ReadMe/Wiki update.

Check which platforms you have built SDK on to verify the correctness of this PR.

  • Linux
  • Windows
  • Android
  • MacOS
  • IOS
  • Other Platforms

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

🤖 Generated with Claude Code

…d memory system

Adds MemoryManagementOptions::sensitiveMemoryManager, Aws::SensitiveAllocator and
Aws::SensitiveString, honoured regardless of USE_AWS_MEMORY_MANAGEMENT. The secret
access key inside Aws::Auth::AWSCredentials and the SigV4 signing key are stored in
Aws::SensitiveString, so an application can keep them out of core dumps.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant