[marcobrambilla99]

Problem. In the parallel mode (fstWriterSetParallelMode(ctx, 1)), a file that is flushed before it is closed can come out broken. Example: 64 variables, 400 time steps, a flush every 5 time steps, and a last time step without changes. Depending on how the threads happen to interleave, the reader either stops with fstReaderIterBlocks2(), fac: 2 clen: 48 (rc=-3), exiting., or reads values that differ from the file the serial writer writes for the same trace. The serial writer is always right.

Cause. In the parallel mode, each flush copies the writer's context and hands the copy to a thread; the writer meanwhile goes on filling the next block. Four things go wrong:

1. fstWriterFlushContextPrivate() copies the context before the previous flush thread has finished. Its lock-and-unlock of the mutex returns at once if the thread has not taken the mutex yet, and its wait on in_pthread comes after the copy. So the copy can still hold the previous section's start.
2. fstWriterClose2() reads the section state after the same lock-and-unlock, so it can also run before the thread has finished.
3. The parallel path clears section_header_only. The serial flush keeps it set after writing the next section's header, so that the close removes a header that no value change follows. In the parallel mode that header is never removed.
4. A flush of a block without value changes, which the serial flush skips, still starts a thread. The thread writes nothing, but the writer counts one more section and starts a new time table.

ThreadSanitizer also reports a data race in the parallel case of tests/write_error.c: the flush thread reads skip_writing_section_hdr in the writer's context while the writer sets flush_context_pending, a bit-field in the same word.

Fix. A new helper, fstWriterWaitFlushThread(), waits for the flush thread by reading in_pthread under the mutex. The thread holds the mutex while it flushes and while it clears in_pthread, so once the wait returns, everything the thread wrote to the writer's context is visible, including the write error added in #34.

• The writer calls it before copying the context, at the start of fstWriterClose2(), and after the close's last flush. The spin loops on in_pthread are removed.
• The parallel path skips a block without value changes, and records section_header_only as the serial flush does.
• The flush thread decides whether to write the next section's header from its own copy of the context, not from the writer's bit-fields.

The serial path is unchanged.

Test. tests/write_parallel.c writes the same pseudo-random trace (variables of 1 to 128 bits, some of them aliases, values 0, 1, x and z) with the serial and with the parallel writer. It reads the parallel file back with fstReaderIterBlocks(), compares every value change with what was written, and then compares the two files byte for byte (both set the same date with fstWriterSetDate()). It covers a flush every 5 time steps and no flush before the close. It avoids blocks without value changes, because the serial writer has its own bug there, fixed in #35. Meson registers the test only when threads are available.

Without this change the new test fails in all 6 builds below. With it, the whole suite passes with gcc 13 and clang 18 on Linux x86-64, plain, with -Db_sanitize=address,undefined, and with -Db_sanitize=thread; under ThreadSanitizer, write_parallel and write_error also passed 20 runs each. Two reports of gcc's UndefinedBehaviorSanitizer are already on main and are not changed here: empty_file and mask_no_variables pass a null pointer with a length of 0 to fwrite() in fstWriterEmitSectionHeader().

AI-assisted (Claude Opus 5.5)

[tbybell]

Please update both this PR and #35 versus what is current in libfst, thanks!

[marcobrambilla99]

Thanks, both #35 and #36 are now updated with current main (195a777).