Repository navigation
Fold mutations of one Search record into one write - #165
Merged
Merged
Conversation
A batch held at most one mutation per resource, so a client that sends several mutations of one record, such as one seller merged once per offer of a search page, waited for each one in turn, and query-visible writes waited for one refresh each. A mutation now says whether its adapter can fold it, and folding mutations of one resource share a batch in queue order, from one stream or several, while a stream's order still holds. Search folds every action but backend expressions. It applies a record's mutations in order to the state of one real-time read, or of none when the first one is a put or delete, and sends one write, conditional on that read. Each mutation keeps its own result: a failed precondition or Lua rejection leaves the state to the next, a conflict evaluates the chain again from a fresh read, and any other rejection runs the mutations one at a time, since none applied. weir_store_folded_mutations_total counts folded mutations.
… fails Re-evaluate a chain without a mutation whose caller left before the write, fail mutations rejected on a state only a failed write would have made real, replay one at a time only when the combined document is rejected, keep folded mutations out of the adaptive limit's batch sizes, and update tests that assumed one native write per increment.
A mutation decided on the record as read that leaves it unchanged keeps APPLIED whatever the chain's write returns, later mutations of a resource no longer join a batch that left out an earlier one, and the docs state that the backend validates only the chain's final document.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A batch held at most one mutation per resource, so mutations of one record waited for each other: in production, PSE merges one seller once per offer of a search page with query-visible Lua transforms, and each merge waited for the previous one's refresh, which showed up as about 200 queue waits over 10 seconds per 35 minutes on pse-search.
Adapters now mark mutations they can fold, and folding mutations of one resource share a batch in queue order while a stream's order still holds. Search folds every action except backend expressions: it applies a record's mutations in order to the state of one real-time read and sends one write conditional on it, so each mutation keeps its own result. Conflicts re-evaluate the chain from a fresh read, and any other rejection replays the mutations one at a time because none applied. MongoDB keeps its current behavior here and follows in a separate change.
weir_store_folded_mutations_totalcounts folded mutations.