Repository navigation
fix: sweep crash leftovers at startup, free a cancelled zip, give each click render its own temp - #900
Merged
Conversation
…h click render its own temp A stem fill, a vocal split decode, a stem written beside its final name, an export render and an all-stems zip each write to a temp name first. A kill or a power loss in the middle left that temp on disk for good, and each can be hundreds of MB. The startup sweep now removes them: only in StemDeck's own song folders, never through a link or junction, never a file a song keeps, and in the export caches only by the exact temp names, and only when nothing has written them for ten minutes (a second server may share the cache). The decode and the zip, with its transcode folder, moved out of the system temp folder (beside the stem, and into the export cache) so the sweep never has to delete in a folder other programs share. A cancelled zip request (the server stopping) skipped the cleanup, because CancelledError is not an Exception, and the build thread went on writing. The zip now goes once the request has given it up and the thread has ended. Two exports rendering the same click at once shared one temp name, so one could rename the other's file away and lose its click. Each render now has its own uuid dot name, removed in a finally.
thcp
added a commit
that referenced
this pull request
Nov 3, 2026
…ed (#901) ## What changes for the user A song whose separation was cut off part way (the app killed, the PC losing power) no longer comes back after a restart as a finished song with stems, the Original track or the mix missing. - A job that was running comes back in the queue and runs again from the start, as a job with no stems on disk already did. - If registry.json was lost as well, an upload comes back as a stopped song, so Extract runs it again from the file it kept. A link's folder stays on disk, untouched, and is not shown as finished. - A song rebuilt from its folder counts as split into Lead + Backing only when both files are there. A kill between the two renames used to bring back a lead vocal without its backing; the lone file stays on disk and the next split replaces it. ## Why restore() rebuilds a song from its folder when the registry says the job was still running, or has no entry for it. It took any folder holding one stem for a finished song. collect() moves the stems in one at a time, and original.wav, mix.wav and the MP3 storage come after it, so a kill at any of those steps returned an incomplete song as "done". #898 made each file arrive whole; this makes the set of files whole. ## How - The last thing a separation writes is `stems/.complete`, after every stem file and the folder are fsynced (`registry.mark_stems_complete`, called at the end of both pipelines). The marker goes through the existing atomic writer: fsynced temp file, rename, folder fsync. - `_rebuild_from_folder` rebuilds a folder as done only when it has the marker. - Songs made before this change: on startup, every song the registry holds as done gets the marker if its folder has none (only the marker is written, the stems are left alone). A song whose registry record was already gone has metadata.json, which StemDeck writes once a song is done (and now only after the marker), and is rebuilt exactly as before. Nothing is hidden, moved or deleted. Songs made by 0.20.0 after a downgrade get the marker on the way back, and 0.20.0 ignores the file. - If the marker cannot be written (a full disk), the job still finishes and the error is logged: failing the job would delete stems that are whole, and the song then rebuilds by its metadata.json like an older song. - Since the marker is the last write, a kill during the lyrics or section analysis after separation now runs the job again, where the old code kept the stems. That is the cost of "never done unless finished". ## Verified - `tests/test_stem_completeness.py` runs the real pipeline steps that write `stems/` (collect, original.wav, mix.wav, MP3 storage) and kills the process at eight points: one stem moved, before original.wav, before mix.wav, after mix.wav, before stem storage, one MP3 moved in, one WAV deleted, after stem storage. Each is checked with the registry kept (job queued again) and lost (the upload comes back stopped with its source). On the old code every one of these came back "done", and the split case listed a lone lead vocal. The finished run is rebuilt as done both ways, and a song from before the marker gets it from the registry. - A real separation on a throwaway server (MP3 storage on): `stems/.complete` is there, and after deleting registry.json a restart brings the song back done with every stem and the mix. - Marker cost, measured on this machine's SSD: 0.14 s for eight 40 MB stems written just before it, 0.004 s when they are already on disk. Not measured on a hard disk or a network share. - Tests whose folders stand for a finished song made by this version now carry the marker. ruff check and ruff format pass. Full Python suite: 3819 passed, 12 skipped; after the rebase onto #899 and #900 the 247 tests of the affected files pass again. Does not touch `uv.lock` or the desktop shell.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes for the user
CancelledErroris not anException), and the build thread went on writing.What the sweep may remove, and what it never touches
registry.job_dirsandholds_job_filesaccept (the rule from fix(reset): remove only StemDeck's own entries in a shared jobs folder #886), and never through a link or junction (is_real_entry)..fill-<8 hex>/(stem fill work folder)stems/:.decode-*/(vocal split decode), and dot names ending in.tmpor.tmp.mp3, file or folder (stem writes)cache/mixdown,cache/click): only the exact temp names (.<name>.<32 hex>.tmp[.ext],.stemdeck_zip_*.zip,.zipwork-*/), and only when nothing has written them for 10 minutes, because a second server can share the cache folder.Other changes that make the sweep exact
finally.tests/conftest.pynow points the export caches at a per-test folder. Every TestClient lifespan runs the sweep, and it must never reach a developer's real cache.How it was verified
tests/test_crash_leftovers.py. All 4 fail onbig-releaseand pass with this change:.DS_Store, and folders that are not StemDeck's.stems/.big-releasethe second one fails withFileNotFoundError; with this change both finish.ruff checkandruff format --checkpass..vocal_split.<hex>.tmpwork folder is covered.Update path
uv.lockor the desktop shell.