Repository navigation
fix(registry): rebuild a song as done only when its separation finished - #901
Merged
Merged
Conversation
restore() rebuilt a "done" song from any folder holding one stem, so a process killed during collect(), before original.wav or mix.wav, or in the middle of MP3 storage came back as a finished song with files missing. A kill between the two vocal-split renames brought back a lead vocal without its backing. The last write of a separation is now stems/.complete, after every stem file and the folder are fsynced, and recovery rebuilds a folder as done only when it has the marker. A folder without it goes back in the queue, or comes back as a stopped upload, and its files are kept. Songs from before the marker get it at startup while the registry holds them as done; one whose record was already lost is still rebuilt by its metadata.json. A rebuilt song counts as split only when both halves of the pair are there.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes for the user
A song whose separation was cut off part way (the app killed, the PC losing power) no longer comes back after a restart as a finished song with stems, the Original track or the mix missing.
Why
restore() rebuilds a song from its folder when the registry says the job was still running, or has no entry for it. It took any folder holding one stem for a finished song. collect() moves the stems in one at a time, and original.wav, mix.wav and the MP3 storage come after it, so a kill at any of those steps returned an incomplete song as "done". #898 made each file arrive whole; this makes the set of files whole.
How
stems/.complete, after every stem file and the folder are fsynced (registry.mark_stems_complete, called at the end of both pipelines). The marker goes through the existing atomic writer: fsynced temp file, rename, folder fsync._rebuild_from_folderrebuilds a folder as done only when it has the marker.Verified
tests/test_stem_completeness.pyruns the real pipeline steps that writestems/(collect, original.wav, mix.wav, MP3 storage) and kills the process at eight points: one stem moved, before original.wav, before mix.wav, after mix.wav, before stem storage, one MP3 moved in, one WAV deleted, after stem storage. Each is checked with the registry kept (job queued again) and lost (the upload comes back stopped with its source). On the old code every one of these came back "done", and the split case listed a lone lead vocal. The finished run is rebuilt as done both ways, and a song from before the marker gets it from the registry.stems/.completeis there, and after deleting registry.json a restart brings the song back done with every stem and the mix.Does not touch
uv.lockor the desktop shell.