Give each Chroma store its own Settings; a shared one crossed the bundles - #220
Merged
Merged
Conversation
…dles
Reactome questions are broken on beta right now. Asked "What does CDK5
phosphorylate in Alzheimer disease?", the deployed container raises:
PermissionError: [Errno 13] Permission denied:
'/app/embeddings/.../userguide/Release95/sections/9c574827-...'
A reactome question, failing inside the user guide bundle -- Chroma trying to
create a segment folder for a collection that is not there, in a bundle it
should never have opened.
`chroma_settings` was one module-level Settings object shared by every store,
and `langchain_chroma` mutates what it is handed:
client_settings.persist_directory = (
persist_directory or client_settings.persist_directory
)
So the last store constructed rewrote the persist directory for every earlier
one, and chromadb returned a client keyed on those settings. With only the
reactome bundle installed this was invisible: every store pointed into the same
tree, so crossing them changed nothing. Installing the user guide bundle
yesterday gave them different trees and turned a latent bug into an outage.
`chroma_settings()` is now a factory returning a fresh object per store.
Three tests, all of which fail against the shared constant. Verified in the
order that broke beta -- user guide first, then reactome, then user guide
again: 6 hits, 40 hits, 6 hits.
**I saw this yesterday and dismissed it.** The same PermissionError appeared
while I was reviewing #216, and I wrote it off as "a local artifact of the
root-owned bundle" after a check that built the two retrievers and queried each
once. That check did not reproduce the ordering, so it proved nothing, and I
reported the system healthy. The bug was in production for a day.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Asked "What does CDK5 phosphorylate in Alzheimer disease?", the deployed container raises:
A reactome question failing inside the user guide bundle — Chroma trying to create a segment folder for a collection that isn't there, in a bundle it should never have opened.
Cause
chroma_settingswas one module-levelSettingsobject shared by every store.langchain_chromamutates what it is handed:So the last store constructed rewrote the persist directory for every earlier one, and chromadb returned a client keyed on those settings.
With only the reactome bundle installed this was invisible — every store pointed into the same tree, so crossing them changed nothing. Installing the user guide bundle yesterday gave them different trees and turned a latent bug into an outage.
chroma_settings()is now a factory returning a fresh object per store.Verified in the order that broke beta
User guide first, then reactome, then user guide again:
Three tests, all of which fail against the shared constant — including one asserting telemetry stays off, so that isn't lost while fixing the sharing.
I saw this yesterday and dismissed it
The same
PermissionErrorappeared while I was reviewing #216. I wrote it off as "a local artifact of the root-owned bundle" after a check that built the two retrievers and queried each once. That check didn't reproduce the ordering, so it proved nothing — and I reported the system healthy. The bug was in production for a day.The check that found it was running the three real question types through the deployed container, which is the one I should have run first.
🤖 Generated with Claude Code