Skip to content

Fix the dump size limit in the parallel writer - #38

Merged
tbybell merged 1 commit into
gtkwave:mainfrom
marcobrambilla99:fix-parallel-size-limit
Nov 9, 2026
Merged

tbybell merged 1 commit into
gtkwave:mainfrom
marcobrambilla99:fix-parallel-size-limit

Conversation

@marcobrambilla99

Copy link
Copy Markdown
Contributor

Problem. In the parallel mode (fstWriterSetParallelMode(ctx, 1)) with a size limit (fstWriterSetDumpSizeLimit()), the reader refuses the file once the limit is reached: Triggered TALOS-2023-1791 security check, exiting. The serial writer writes a good file for the same trace.

Cause. Two things go wrong when a flush thread reaches the limit:

  1. The thread sets skip_writing_section_hdr, size_limit_locked and is_initial_time in the writer's context. These are bit-fields that share a word with flags the writer changes at the same time, so either side can lose the other's update.
  2. The writer goes on adding value changes to the next block. The serial writer drops them, because it stops at the flush that reaches the limit.

Fix. The flush thread no longer writes the writer's bit-fields. Under the mutex it records only that it reached the limit (a byte of its own, size_limit_reached_in_thread) and the time step at which its flush started. It decides about the next section header from its own copy of the context. The writer applies the stop in a new function, fstWriterApplyThreadSizeLimit(), when it next waits for the thread (at its next flush or at the close):

  • it sets the three flags, as the serial flush does;
  • it drops the value changes and time steps added since that flush;
  • it sets the end time back to the time step of that flush, and moves the blackouts (fstWriterEmitDumpActive()) recorded after it to that time, where the serial writer records them;
  • the time change that applies the stop returns at once, because the serial writer ignores every time change after its stop.

fstWriterGetDumpSizeLimitReached() also reports a stop that a finished thread has not handed over yet. It reads the thread's byte with pthread_mutex_trylock(), so it neither races with a running thread nor waits for it.

Test. tests/write_parallel.c gains two cases: a size limit of 60 000 bytes on a trace that reaches it after a few flushes, with and without blackouts. The parallel writer must write the serial writer's file byte for byte, and the value changes read back must be the first ones written. On main the new cases fail with the message above; with this change the suite passes with gcc 13 and clang 18, also with AddressSanitizer and UndefinedBehaviorSanitizer and with ThreadSanitizer (write_parallel and write_error 20 runs each), apart from two UndefinedBehaviorSanitizer reports that main already has (empty_file and mask_no_variables pass a null pointer with a length of 0 to fwrite()).

A wider comparison wrote traces of 64 and 300 variables with size limits from 20 000 to 600 000 bytes, with and without blackouts, and traces of up to 3 000 variables without a limit, in the serial and the parallel mode: every file was accepted by the reader, the changes read back were the first ones written (all of them where the file stayed under its limit), and the parallel writer's file was the serial writer's file byte for byte.

AI-assisted (Claude Opus 5.5)

Problem: with fstWriterSetParallelMode(ctx, 1) and
fstWriterSetDumpSizeLimit(), the reader refuses the file ("TALOS-2023-1791
security check"). The flush thread that reaches the limit sets
skip_writing_section_hdr, size_limit_locked and is_initial_time in the
writer's context. These are bit-fields that share a word with flags the
writer changes at the same time, and the writer goes on emitting into a
block that the serial writer would have dropped.

Solution: the flush thread sets a byte of its own,
size_limit_reached_in_thread, and the time step at which its flush was
started, under the mutex, and decides about the next section header from
its own copy of the context. The writer applies the stop once the thread
is done, at its next flush or at the close: it sets the three flags, drops
what it emitted since, sets the end time back to the time step of that
flush and moves the blackouts recorded after it to it, as the serial writer
records them. A time change that applies the stop returns at once.
fstWriterGetDumpSizeLimitReached() also reports a stop that a finished
thread has not handed over yet; it does not wait for a running thread.

tests/write_parallel.c checks that, under a size limit, with and without
blackouts, the parallel writer writes the serial writer's file and that the
changes read back are the first ones written.

Signed-off-by: Marco Brambilla <marco@hairyotter.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tbybell
tbybell merged commit 8f60a7e into gtkwave:main Nov 9, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants