[bugfix] decrypt pages outside the AIO array mutex - #114
Open
puli2300 wants to merge 1 commit into
Open
Conversation
In the Linux native AIO completion path, find_completed_slot() returned the completed slot while still holding the AIO array mutex, so the whole completion processing - check_state(), post_io_processing(), io_complete() and, for encrypted or compressed tablespaces, the page decryption and decompression - ran under that mutex. Every I/O handler thread and every submitter (AIO::reserve_slot() needs the same mutex) serialized behind it, capping completion throughput at one page-processing time regardless of the number of handler threads. With AES-NI a 16 KiB page decrypts in ~16 us and the damage stays invisible, but with a slower software cipher implementation (~40 us per page) aggregate throughput saturates at ~40 MB/s no matter how many handler threads run, and page cleaner write-out stalls behind read completions as well. Fix: find_completed_slot() now claims the slot under the mutex by clearing io_already_done - the kernel event was already reaped by collect(), so the slot cannot be marked again, and other handler threads skip slots with io_already_done == false - and releases the mutex before returning. The caller then exclusively owns the slot: completion processing, including page decryption, runs without the mutex held. poll() re-acquires the mutex only to resubmit a partially completed request and to free the slot via AIO::release(). This matches the existing patterns: the write path already performs page compression and encryption outside the array mutex (AIO::reserve_slot()), and the simulated AIO handler performs its completion processing without holding the mutex. Measured on the same code base (MySQL 8.0.25 lineage) with a cold full-table-scan workload over encrypted tablespaces on a RAM disk: throughput with a software cipher went from 0.038 GB/s to 1.18 GB/s at 8 handler threads (31x), AES improved 3x, and unencrypted throughput was unchanged. Old and new servers read identical data (full-table CRC32 over an encrypted tablespace), and the MTR encryption suite passes fully on a debug build.
|
lipu seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #113
In the Linux native AIO completion path, find_completed_slot() returned the completed slot while still holding the AIO array mutex, so the whole completion processing - check_state(), post_io_processing(), io_complete() and, for encrypted or compressed tablespaces, the page decryption and decompression - ran under that mutex. Every I/O handler thread and every submitter (AIO::reserve_slot() needs the same mutex) serialized behind it, capping completion throughput at one page-processing time regardless of the number of handler threads.
With AES-NI a 16 KiB page decrypts in ~16 us and the damage stays invisible, but with a slower software cipher implementation (~40 us per page) aggregate throughput saturates at ~40 MB/s no matter how many handler threads run, and page cleaner write-out stalls behind read completions as well.
Fix: find_completed_slot() now claims the slot under the mutex by clearing io_already_done - the kernel event was already reaped by collect(), so the slot cannot be marked again, and other handler threads skip slots with io_already_done == false - and releases the mutex before returning. The caller then exclusively owns the slot: completion processing, including page decryption, runs without the mutex held. poll() re-acquires the mutex only to resubmit a partially completed request and to free the slot via AIO::release().
This matches the existing patterns: the write path already performs page compression and encryption outside the array mutex (AIO::reserve_slot()), and the simulated AIO handler performs its completion processing without holding the mutex.
Measured on the same code base (MySQL 8.0.25 lineage) with a cold full-table-scan workload over encrypted tablespaces on a RAM disk: throughput with a software cipher went from 0.038 GB/s to 1.18 GB/s at 8 handler threads (31x), AES improved 3x, and unencrypted throughput was unchanged. Old and new servers read identical data (full-table CRC32 over an encrypted tablespace), and the MTR encryption suite passes fully on a debug build.