Conversation
Registration records the size by stat'ing the lease's MetadataPath. On Linux and macOS that is a descriptor path, so the row gets the descriptor link's own 64 bytes instead of the audio file's length. Nothing in the suite noticed: AudioFileServiceTests asserted Size nowhere, and the one lease double it has sets MetadataPath equal to PublicPath, which is the Windows shape, where the two paths agree and the bug cannot appear. The new double gives MetadataPath its own 64-byte file, matching what stat reports for a /proc/<pid>/fd/<n> link, and covers both lease shapes: one that serves a generation-bound read stream and one that does not. Both register a 12,345 byte file and both currently record 64. DivergentDescriptorFixture_ReportsDescriptorLengthRatherThanAudioLength is the control. Point the fixture's two paths at one file and the registration tests go green without exercising anything, so that test asserts the divergence the other two depend on. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…r path EnsureAudiobookFileCoreAsync took the size by stat'ing metadataPath, which is the lease's MetadataPath whenever a lease is present. On Linux and macOS that is a descriptor path, /proc/<pid>/fd/<n> or /dev/fd/<n>, so stat reports the descriptor link's own size rather than the length of the file it pins. Every row created through this path on Linux recorded 64 bytes, which is what stat returns for a procfs fd link. Windows was unaffected, because the pinned lease sets MetadataPath to the canonical path there and the two agree. The value was already in scope: filePath is handed to ExtractMetadataAsync five lines above as the public half of the read. Size now comes from the lease's own generation-bound read stream, which is served from the pinned descriptor and reports the pinned file's length. That keeps the guarantee the lease exists to provide, since it never consults the visible path, and it matches how FileRegistrationRecoveryService already checks a published file against its journalled length. Leases exposing no generation-bound read, and registrations with no lease at all, fall back to the published path through IFileSystem. Reported and diagnosed by kevinroberts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Closing this in favour of #901, which is the earlier and the more complete fix. @kevinheneveld opened #901 on 25 August against the same Two things here that #901 does not have, in case either is wanted: three tests that pin the stored length against a literal rather than against another read of the same value, and a refinement that takes the length from the lease's own stream where the lease exposes one. Happy to put either on #901 as a review comment if that is useful, or to leave it alone. The branch stays on my fork, so if #901 is ever abandoned this can come back in one push. Disclosure: drafted with Claude Code at my direction; I read the cited code at the stated commit and reviewed this before posting. |
Fixes #821.
The diagnosis in that issue is kevinroberts's and it holds up. He found the call site, the line, the mechanism and the regression window before anyone else looked at it. This is his fix with a test around it.
What was wrong
EnsureAudiobookFileCoreAsyncresolves a metadata path atAudiobookFileService.cs:173:and then, 157 lines later, records the size from it:
On Linux and macOS a lease's
MetadataPathis a descriptor path. It is built atPinnedAudiobookFileRegistrationLease.cs:151-154, again at:218-219, and atPathOnlyAudiobookFileRegistrationLease.cs:36-39. Stat one of those and you get the size of the descriptor link, not the length of the file the descriptor pins. Windows escapes it because the same lease setsMetadataPathto the canonical path there (PinnedAudiobookFileRegistrationLease.cs:138), so the two paths agree.The value it needed was already sitting in scope.
filePathgoes toExtractMetadataAsyncas the public half of the read at:325-328, five lines above theFileInfothat gets it wrong.Where it bites
EnsureAudiobookFileCoreAsyncis the row-creation path, and every caller reaching it passes a lease:AudiobookScanService.cs:279AudiobookScanService.Reconciliation.cs:349AudiobookScanService.ExistingRegistration.cs:136RegisterPublishedGenerationAsync, which routes into the same core atAudiobookFileService.Registration.cs:107,:113,:181and:187Registration without a lease was always fine, since
:173falls back tofilePath. That fits kevinroberts seeing every row affected rather than a subset: on Linux all of those routes carry a lease.The fix
Size now comes from the lease's own read stream, which is served from the pinned descriptor and reports the pinned file's length. Since it never consults the visible path, the guarantee the lease exists to provide survives.
FileRegistrationRecoveryService.Receipts.cs:86-87already treats that stream'sLengthas authoritative when it checks a published file against its journalled length, so this is the pattern the codebase had already settled on.Leases exposing no generation-bound read, and registrations with no lease at all, fall back to the published path through
IFileSystem.GetFileLength.Nothing in the interfaces changed and no new lease member was added.
OpenMetadataReadStreamis already onIAudiobookFileRegistrationLease.Evidence
Measured. A .NET 10 console program writes a 1,048,576 byte file and stats both paths:
The first line is the control. Same program, same file, real path, right answer, which is what rules out the harness as the source of the 64. So the reported 64 is a property of the descriptor path rather than of anybody's audio files.
It reproduces in the suite too. Before the fix, both new registration tests fail with
Expected: 12345, Actual: 64.Tests
AudioFileServiceTestsassertedSizenowhere at all before this, and the one lease double it had setMetadataPathequal toPublicPath. That is the Windows shape, where the two paths agree and the fault cannot show up, so a green suite was never saying anything about this.Three tests, all platform-agnostic so they run on both CI legs:
EnsureAudiobookFileAsync_LeaseExposingGenerationBoundRead_RecordsAudioFileSizeEnsureAudiobookFileAsync_LeaseWithoutGenerationBoundRead_RecordsAudioFileSizeDivergentDescriptorFixture_ReportsDescriptorLengthRatherThanAudioLengthThe new double gives
MetadataPathits own 64-byte file, matching what stat reports for a procfs fd link, and covers both lease shapes: one serving a generation-bound read, one reproducing the interface default and throwing.The third test is the control. Point the fixture's two paths at a single file and the other two go green while exercising nothing, so that test asserts the divergence they depend on. The two commits are split for the same reason. Check out the first one and the registration tests fail, which is the part I would rather you could verify than take on trust.
Suite, measured on this branch:
a630572e0255f9a2The three added are the three new tests. Nothing else changed status.
What this does not claim, and one site left alone
The second site kevinroberts credits me with has the same shape, in
CreatePhysicalGenerationSnapshot. Ata630572eit sits atAudiobookFileService.PhysicalGeneration.cs:277, assigned at:280. The issue cites:271and:274, which were its lines back atf27c7989.I have deliberately left it alone. That file is being rewritten in #993, which swaps
ClonePhysicalGenerationforCapturePhysicalGenerationand reworks the restore calls around it. Patching the same file underneath an open rewrite seemed like a bad trade for a one-line change, so I would rather it went in after #993 lands, or inside it if that is easier for you. The helper here is callable from that file, so the follow-up is one line. Happy to add it to this PR instead if you would prefer that.Two things I looked at and am not claiming as part of this:
ExtractMetadataAsyncbuilds its cache key fromnew FileInfo(metadataPath).LastWriteTimeUtc.Ticks, atAudiobookFileService.MetadataExtraction.cs:16-18and:71-73. On Linux that reads the descriptor link's timestamp rather than the file's. Read from the source, not measured, and out of scope here.AudiobookFileService.MetadataRefresh.csdoes not setSize, so it does not reintroduce this.Not measured on a running instance. There is no runtime behaviour to watch here beyond the recorded value, and the tests pin that, but I have not sat and watched a real library rescan on a build carrying this.
Disclosure: drafted with Claude Code at my direction; I read the cited code at the stated commit and reviewed this before posting.