Repository navigation
Fix the dump size limit in the parallel writer - #38
Merged
Merged
Conversation
Problem: with fstWriterSetParallelMode(ctx, 1) and
fstWriterSetDumpSizeLimit(), the reader refuses the file ("TALOS-2023-1791
security check"). The flush thread that reaches the limit sets
skip_writing_section_hdr, size_limit_locked and is_initial_time in the
writer's context. These are bit-fields that share a word with flags the
writer changes at the same time, and the writer goes on emitting into a
block that the serial writer would have dropped.
Solution: the flush thread sets a byte of its own,
size_limit_reached_in_thread, and the time step at which its flush was
started, under the mutex, and decides about the next section header from
its own copy of the context. The writer applies the stop once the thread
is done, at its next flush or at the close: it sets the three flags, drops
what it emitted since, sets the end time back to the time step of that
flush and moves the blackouts recorded after it to it, as the serial writer
records them. A time change that applies the stop returns at once.
fstWriterGetDumpSizeLimitReached() also reports a stop that a finished
thread has not handed over yet; it does not wait for a running thread.
tests/write_parallel.c checks that, under a size limit, with and without
blackouts, the parallel writer writes the serial writer's file and that the
changes read back are the first ones written.
Signed-off-by: Marco Brambilla <marco@hairyotter.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem. In the parallel mode (
fstWriterSetParallelMode(ctx, 1)) with a size limit (fstWriterSetDumpSizeLimit()), the reader refuses the file once the limit is reached:Triggered TALOS-2023-1791 security check, exiting.The serial writer writes a good file for the same trace.Cause. Two things go wrong when a flush thread reaches the limit:
skip_writing_section_hdr,size_limit_lockedandis_initial_timein the writer's context. These are bit-fields that share a word with flags the writer changes at the same time, so either side can lose the other's update.Fix. The flush thread no longer writes the writer's bit-fields. Under the mutex it records only that it reached the limit (a byte of its own,
size_limit_reached_in_thread) and the time step at which its flush started. It decides about the next section header from its own copy of the context. The writer applies the stop in a new function,fstWriterApplyThreadSizeLimit(), when it next waits for the thread (at its next flush or at the close):fstWriterEmitDumpActive()) recorded after it to that time, where the serial writer records them;fstWriterGetDumpSizeLimitReached()also reports a stop that a finished thread has not handed over yet. It reads the thread's byte withpthread_mutex_trylock(), so it neither races with a running thread nor waits for it.Test.
tests/write_parallel.cgains two cases: a size limit of 60 000 bytes on a trace that reaches it after a few flushes, with and without blackouts. The parallel writer must write the serial writer's file byte for byte, and the value changes read back must be the first ones written. On main the new cases fail with the message above; with this change the suite passes with gcc 13 and clang 18, also with AddressSanitizer and UndefinedBehaviorSanitizer and with ThreadSanitizer (write_parallelandwrite_error20 runs each), apart from two UndefinedBehaviorSanitizer reports that main already has (empty_fileandmask_no_variablespass a null pointer with a length of 0 tofwrite()).A wider comparison wrote traces of 64 and 300 variables with size limits from 20 000 to 600 000 bytes, with and without blackouts, and traces of up to 3 000 variables without a limit, in the serial and the parallel mode: every file was accepted by the reader, the changes read back were the first ones written (all of them where the file stayed under its limit), and the parallel writer's file was the serial writer's file byte for byte.
AI-assisted (Claude Opus 5.5)