Skip to content

Record AudiobookFile.Size from the audio file rather than the lease descriptor path - #997

Closed
m4bard wants to merge 2 commits into
Listenarrs:canaryfrom
m4bard:fix/audiobook-file-size-from-real-path
Closed

m4bard wants to merge 2 commits into
Listenarrs:canaryfrom
m4bard:fix/audiobook-file-size-from-real-path

Conversation

@m4bard

@m4bard m4bard commented Sep 18, 2026

Copy link
Copy Markdown

Fixes #821.

The diagnosis in that issue is kevinroberts's and it holds up. He found the call site, the line, the mechanism and the regression window before anyone else looked at it. This is his fix with a test around it.

What was wrong

EnsureAudiobookFileCoreAsync resolves a metadata path at AudiobookFileService.cs:173:

var metadataPath = registrationLease?.MetadataPath ?? filePath;

and then, 157 lines later, records the size from it:

var fi = new FileInfo(metadataPath);          // :330
var fileRecord = AudiobookFile.CreateUnresolved(filePath);
fileRecord.AudiobookId = audiobook.Id;
fileRecord.Size = fi.Exists ? fi.Length : null;   // :333

On Linux and macOS a lease's MetadataPath is a descriptor path. It is built at PinnedAudiobookFileRegistrationLease.cs:151-154, again at :218-219, and at PathOnlyAudiobookFileRegistrationLease.cs:36-39. Stat one of those and you get the size of the descriptor link, not the length of the file the descriptor pins. Windows escapes it because the same lease sets MetadataPath to the canonical path there (PinnedAudiobookFileRegistrationLease.cs:138), so the two paths agree.

The value it needed was already sitting in scope. filePath goes to ExtractMetadataAsync as the public half of the read at :325-328, five lines above the FileInfo that gets it wrong.

Where it bites

EnsureAudiobookFileCoreAsync is the row-creation path, and every caller reaching it passes a lease:

  • library scan, at AudiobookScanService.cs:279
  • legacy reconciliation, at AudiobookScanService.Reconciliation.cs:349
  • existing-file registration, at AudiobookScanService.ExistingRegistration.cs:136
  • download import and manual import, through RegisterPublishedGenerationAsync, which routes into the same core at AudiobookFileService.Registration.cs:107, :113, :181 and :187

Registration without a lease was always fine, since :173 falls back to filePath. That fits kevinroberts seeing every row affected rather than a subset: on Linux all of those routes carry a lease.

The fix

Size now comes from the lease's own read stream, which is served from the pinned descriptor and reports the pinned file's length. Since it never consults the visible path, the guarantee the lease exists to provide survives. FileRegistrationRecoveryService.Receipts.cs:86-87 already treats that stream's Length as authoritative when it checks a published file against its journalled length, so this is the pattern the codebase had already settled on.

Leases exposing no generation-bound read, and registrations with no lease at all, fall back to the published path through IFileSystem.GetFileLength.

Nothing in the interfaces changed and no new lease member was added. OpenMetadataReadStream is already on IAudiobookFileRegistrationLease.

Evidence

Measured. A .NET 10 console program writes a 1,048,576 byte file and stats both paths:

CONTROL FileInfo(real path)     : Length=1048576
FileInfo(/proc/<pid>/fd/<n>)    : Length=64
RandomAccess.GetLength(handle)  : 1048576
FileStream(handle).Length       : 1048576

The first line is the control. Same program, same file, real path, right answer, which is what rules out the harness as the source of the 64. So the reported 64 is a property of the descriptor path rather than of anybody's audio files.

It reproduces in the suite too. Before the fix, both new registration tests fail with Expected: 12345, Actual: 64.

Tests

AudioFileServiceTests asserted Size nowhere at all before this, and the one lease double it had set MetadataPath equal to PublicPath. That is the Windows shape, where the two paths agree and the fault cannot show up, so a green suite was never saying anything about this.

Three tests, all platform-agnostic so they run on both CI legs:

  • EnsureAudiobookFileAsync_LeaseExposingGenerationBoundRead_RecordsAudioFileSize
  • EnsureAudiobookFileAsync_LeaseWithoutGenerationBoundRead_RecordsAudioFileSize
  • DivergentDescriptorFixture_ReportsDescriptorLengthRatherThanAudioLength

The new double gives MetadataPath its own 64-byte file, matching what stat reports for a procfs fd link, and covers both lease shapes: one serving a generation-bound read, one reproducing the interface default and throwing.

The third test is the control. Point the fixture's two paths at a single file and the other two go green while exercising nothing, so that test asserts the divergence they depend on. The two commits are split for the same reason. Check out the first one and the registration tests fail, which is the part I would rather you could verify than take on trust.

Suite, measured on this branch:

Passed Failed Skipped Total
a630572e 3118 0 130 3248
0255f9a2 3121 0 130 3251

The three added are the three new tests. Nothing else changed status.

What this does not claim, and one site left alone

The second site kevinroberts credits me with has the same shape, in CreatePhysicalGenerationSnapshot. At a630572e it sits at AudiobookFileService.PhysicalGeneration.cs:277, assigned at :280. The issue cites :271 and :274, which were its lines back at f27c7989.

I have deliberately left it alone. That file is being rewritten in #993, which swaps ClonePhysicalGeneration for CapturePhysicalGeneration and reworks the restore calls around it. Patching the same file underneath an open rewrite seemed like a bad trade for a one-line change, so I would rather it went in after #993 lands, or inside it if that is easier for you. The helper here is callable from that file, so the follow-up is one line. Happy to add it to this PR instead if you would prefer that.

Two things I looked at and am not claiming as part of this:

  • ExtractMetadataAsync builds its cache key from new FileInfo(metadataPath).LastWriteTimeUtc.Ticks, at AudiobookFileService.MetadataExtraction.cs:16-18 and :71-73. On Linux that reads the descriptor link's timestamp rather than the file's. Read from the source, not measured, and out of scope here.
  • Fix weak-storage recovery and root confirmation #993's new AudiobookFileService.MetadataRefresh.cs does not set Size, so it does not reintroduce this.

Not measured on a running instance. There is no runtime behaviour to watch here beyond the recorded value, and the tests pin that, but I have not sat and watched a real library rescan on a build carrying this.

Disclosure: drafted with Claude Code at my direction; I read the cited code at the stated commit and reviewed this before posting.

m4bard and others added 2 commits September 17, 2026 23:17
Registration records the size by stat'ing the lease's MetadataPath. On Linux
and macOS that is a descriptor path, so the row gets the descriptor link's own
64 bytes instead of the audio file's length. Nothing in the suite noticed:
AudioFileServiceTests asserted Size nowhere, and the one lease double it has
sets MetadataPath equal to PublicPath, which is the Windows shape, where the
two paths agree and the bug cannot appear.

The new double gives MetadataPath its own 64-byte file, matching what stat
reports for a /proc/<pid>/fd/<n> link, and covers both lease shapes: one that
serves a generation-bound read stream and one that does not. Both register a
12,345 byte file and both currently record 64.

DivergentDescriptorFixture_ReportsDescriptorLengthRatherThanAudioLength is the
control. Point the fixture's two paths at one file and the registration tests
go green without exercising anything, so that test asserts the divergence the
other two depend on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…r path

EnsureAudiobookFileCoreAsync took the size by stat'ing metadataPath, which is
the lease's MetadataPath whenever a lease is present. On Linux and macOS that
is a descriptor path, /proc/<pid>/fd/<n> or /dev/fd/<n>, so stat reports the
descriptor link's own size rather than the length of the file it pins. Every
row created through this path on Linux recorded 64 bytes, which is what stat
returns for a procfs fd link. Windows was unaffected, because the pinned lease
sets MetadataPath to the canonical path there and the two agree.

The value was already in scope: filePath is handed to ExtractMetadataAsync five
lines above as the public half of the read.

Size now comes from the lease's own generation-bound read stream, which is
served from the pinned descriptor and reports the pinned file's length. That
keeps the guarantee the lease exists to provide, since it never consults the
visible path, and it matches how FileRegistrationRecoveryService already checks
a published file against its journalled length. Leases exposing no
generation-bound read, and registrations with no lease at all, fall back to the
published path through IFileSystem.

Reported and diagnosed by kevinroberts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@m4bard
m4bard requested a review from a team September 18, 2026 04:43
@m4bard

m4bard commented Sep 19, 2026

Copy link
Copy Markdown
Author

Closing this in favour of #901, which is the earlier and the more complete fix.

@kevinheneveld opened #901 on 25 August against the same new FileInfo(...) length read on the lease metadata path, and it covers both call sites. This one deferred the second site in AudiobookFileService.PhysicalGeneration.cs until after #993, so #901 is strictly ahead. I should have found it before opening this, and I did not.

Two things here that #901 does not have, in case either is wanted: three tests that pin the stored length against a literal rather than against another read of the same value, and a refinement that takes the length from the lease's own stream where the lease exposes one. Happy to put either on #901 as a review comment if that is useful, or to leave it alone.

The branch stays on my fork, so if #901 is ever abandoned this can come back in one push.

Disclosure: drafted with Claude Code at my direction; I read the cited code at the stated commit and reviewed this before posting.

@m4bard m4bard closed this Sep 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Every AudiobookFiles row records Size = 64FileInfo stats the /proc fd path instead of the file

1 participant