[CLIENT-681] Optimize performance when using bytes across the client API by reducing the number of heap allocations - #779
Open
DomPeliniAerospike wants to merge 83 commits into
Open
[CLIENT-681] Optimize performance when using bytes across the client API by reducing the number of heap allocations#779DomPeliniAerospike wants to merge 83 commits into
DomPeliniAerospike wants to merge 83 commits into
Conversation
* Replace static_pool with dynamic_pool CLIENT-681 Replaced static_pool with dynamic_pool Change as_bytes from heap to stack for operations and expressions. * Added an expanding block size and pointer block CLIENT-681 Added an expanding block size and pointer block to avoid several realloc calls * CLIENT-681 Change Dynamic_pool_resize, dynamic_pool_init, and pool_destroy to inline functions rather than macros Ran linter and fixed flake issues Renamed dynamic_pool.h to dynamic_pool.hpp to solve precommit issues * Change dynamic_pool.h to dynamic_pool.hpp To fix clang-format issues * Replace static_pool with dynamic_pool CLIENT-681 Replaced static_pool with dynamic_pool Change as_bytes from heap to stack for operations and expressions. * Added an expanding block size and pointer block CLIENT-681 Added an expanding block size and pointer block to avoid several realloc calls * CLIENT-681 Change Dynamic_pool_resize, dynamic_pool_init, and pool_destroy to inline functions rather than macros Ran linter and fixed flake issues Renamed dynamic_pool.h to dynamic_pool.hpp to solve precommit issues * Change dynamic_pool.h to dynamic_pool.hpp To fix clang-format issues * Update cdt_list_operate.c * changed pool_destroy to true for querys and scans changed pool_destroy to true for querys and scans * Testing error conditions * FF to Dev (#620) * [CLIENT-1837] CI/CD: Label version with git commit when building on a non-tagged commit (#605) * Also label version when building unoptimized, with macOS debug support, or with uncommitted changes to the repo (i.e add a "dirty" label) * Rename config.conf to config.conf.template and add config.conf to .gitignore, so creating and modifying config.conf won't cause the repo to be in a dirty state * Auto-bump version to 15.0.1rc3.dev1 [skip ci] * [DEVOPS-234] CI/CD: Build manylinux wheels and upload them to JFrog on every push to an arbitrary branch (#606) * CI/CD: Add option to build and upload unoptimized wheels to JFrog for QE * Auto-bump version to 15.0.1rc3.dev2 [skip ci] --------- Co-authored-by: juliannguyen4 <109386615+juliannguyen4@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Refactored CLIENT-681 Refactored dynamic_pool.hpp to increase readability. Fixed memory bug with dynamic_pool in queries and scans. Added allocate_buffer as a variable to necessary functions. * Update test_query_apply.py * Update dynamic_pool.hpp * Update dynamic_pool.hpp * Corrected dynamic_pool * Update apply.c * Update type.c * Resetting state * Use manylinux builds for valgrind tests * oop * Update serializer.c * Fixed as_bytes_init_wrap to use heap_b when needed * Fixed memory leak in serializer.c removed allocate_buffer parameter from serialize_based_on_serializer_policy. Added true for freeBytes in destroy_dynamic_pool in put call. * Update serializer.c * Update serializer.c * fixed calloc typo * Modified serializer to only get bytes from bytes pool if there is no error --------- Co-authored-by: juliannguyen4 <109386615+juliannguyen4@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Change the descriptive units Added documentation Removed redundant code
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## dev #779 +/- ##
==========================================
+ Coverage 85.46% 85.59% +0.13%
==========================================
Files 101 102 +1
Lines 14661 14832 +171
==========================================
+ Hits 12530 12696 +166
- Misses 2131 2136 +5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Made put serializer more consistent Removed SERIALIZER_PYTHON usage Simplified serializer code Readded serialization in places to match prior usage.
…ors don't get ignored
…trictypes() is called, as_operations ops's number of operations is manually incremented, but the as_binop's bin field is undefined.
…(that is the first function to be called)
…er as_* helper naming
…ent with the C client's naming
…_dynamic_pool_free_table to as_dynamic_pool_destroy. TODO - risk is as_dynamic_pool_destroy may not be inlined because the amount of code in it has increased.
…o it isn't copied over to other source files
… in aerospike.c, but not defined in that same file. Removing static allows as_dynamic_pool_get_as_bytes to be defined in another source file i.e dynamic_pool.c
… as well. Since we moved as_dynamic_pool_get_as_bytes's definition to a source file, don't make it inline anymore (here, we trust the compiler to optimize this function properly)
… var, we need to define as_dynamic_pool in the header file so the struct size is known
… Add missing decl for as_dynamic_pool_destroy
Collaborator
|
Merge this after 8.1.3-compatible client release to reduce chances of these large changes causing issues with the client release |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix bug where only up to 4096 bytearray objects can be used in one client command
Replaces the static pool with a dynamically allocated byte pool.
TODO
Additional tickets:
CLIENT-3501