Repository navigation
Report a failed write from the writer - #34
Conversation
Problem: when a write to the FST file or one of its working files fails, for example on a full disk, the writer does not notice. fstWriterClose() returns as usual and the file is left incomplete, and the caller has no way to find out. Solution: keep the errno of the first write that fails. Every flush, the final fclose(), the gzip writes and the rename in fstWriterClose() are checked; stdio keeps a stream's error indicator set, so a check at each flush also catches a failed write since the last one. In parallel mode the flush thread passes its failure to the parent context before its copy is freed. A new query, fstWriterGetWriteError(), answers it while the file is open, as fstWriterGetFseekFailed() does for a seek, and fstWriterClose() now returns it, or 0, as fclose() and gzclose() return a status. Existing calls of fstWriterClose() still compile. The new test makes writes fail with RLIMIT_FSIZE, so it is skipped on Windows.
|
Please don't change the type of fstWriterClose, mainly because I don't want to cause potential compiler warnings with future scanners. |
|
Thanks. I'll keep fstWriterClose as it is and add a separate function that returns the status. It closes the file the same way and answers the errno of the first write that failed, or 0. fstWriterClose then calls it and ignores the result. The name follows fstWriterCreateVar2. Is that name OK, or would you prefer another one? Also, what do you mean by future scanners? Is it tools that read fstapi.h, like the language bindings, or something else? |
|
By future scanners, I mean compilers, etc. that might fail when warnings levels are turned to maximum. |
|
The configuration 'macos-14-arm64-us-default' is not supported now appears. I'm going to bypass the check for now on the PR. |
Fixes #33.
Problem. When a write to the FST file or one of its working files fails, for example on a full disk, the writer does not notice.
fstWriterClose()returns as usual and the file is left incomplete, and the caller has no way to find out.Solution. Keep the errno of the first write that fails, and answer it in two places.
fclose(), the gzip writes and the rename in the close are checked. stdio keeps a stream's error indicator set, so a check at each flush also catches a failed write since the last one.fstWriterGetWriteError()followsfstWriterGetFseekFailed().fstWriterClose()keeps its type and callsfstWriterClose2(), which returns a status asfclose()andgzclose()do.tests/write_error.cmakes writes fail withRLIMIT_FSIZE. It covers a file written whole, one that fails while it is written (also in parallel mode), and one that fails only in the writes that end it. It is skipped on Windows, which has no such limit.