Repository navigation
Reduce per-container memory: cache by GUID instead of Container - #1287
Merged
Merged
Conversation
labkey-matthewb
approved these changes
Oct 5, 2026
labkey-jeckels
changed the title
labkey-jeckels
added a commit
to LabKey/platform
that referenced
this pull request
Oct 10, 2026
…e JSON keys (#8134) ## Tasks - [x] Claude Code Review - [x] Manual Testing - N/A - [x] Test Automation - [x] Check TeamCity Suites - [x] [Daily Postgres](https://teamcity.labkey.org/build/4203956) - [x] [Nightly SM/FM postgres](https://teamcity.labkey.org/build/4203972) --- ## Rationale Lower the server's memory footprint on servers with many containers and concurrent bulk JSON API requests. A heap dump from a production server with about 74,000 workbooks and 41 concurrent bulk list inserts showed two main costs: | Source | In the dump | Estimated after this PR | |---|---|---| | Bulk JSON payloads: the parse trees of 41 concurrent `insertRows` requests | 1.88 GB, about 108 B per cell | about half of that, ~0.95 GB saved, because each distinct key is stored once per request instead of once per row | | Caches holding one entry per container | 0.39 GB across all caches | ~175–230 MB saved | | `Container` objects | 194,496 for 74,338 containers; stale copies are kept alive as keys in Container-keyed caches | one per container | Savings are estimates from object layouts and the dump's entry counts, not measurements of the patched code. BVT-EHR currently runs develop's specimen module rather than this branch's, so expect container-delete failures there until [GH Issue 1643](LabKey/internal-issues#1643) is resolved. ## Related Pull Requests - LabKey/limsModules#2516 - LabKey/premiumModules#776 - LabKey/customModules#282 - LabKey/targetedms#1287 ## Changes - Container-scoped caches use the container's GUID as their key instead of the `Container` object, and the cache layer now rejects `Container` keys - Container cache invalidation removes only the affected entries instead of clearing whole caches, and reuses cached `Container` objects - The folder-type and reports usage metrics and the required-modules calculation query the database directly instead of filling a cache entry for every container - The pipeline-root and file-root tables are each cached as a single map, and the WebDAV folder cache is capped at a fixed size - The search crawler indexes only a workbook's own lists, not its parent's - Bulk JSON request parsing stores one copy of each repeated object key - Optimize some metrics (folder type counts, etc) to do a single SQL query to get results instead of iterating all containers --------- Co-authored-by: Adam Rauch <adam@labkey.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale
Companion to LabKey/platform#8134, which rejects
Containerobjects as cache keys to reduce per-container memory.Related Pull Requests
Changes
Containerobject