Skip to content

fix(registry): rebuild a song as done only when its separation finished - #901

Merged
thcp merged 1 commit into
big-releasefrom
fix/b40b-stem-completeness-marker
Nov 3, 2026
Merged

thcp merged 1 commit into
big-releasefrom
fix/b40b-stem-completeness-marker

Conversation

@thcp

@thcp thcp commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

What changes for the user

A song whose separation was cut off part way (the app killed, the PC losing power) no longer comes back after a restart as a finished song with stems, the Original track or the mix missing.

  • A job that was running comes back in the queue and runs again from the start, as a job with no stems on disk already did.
  • If registry.json was lost as well, an upload comes back as a stopped song, so Extract runs it again from the file it kept. A link's folder stays on disk, untouched, and is not shown as finished.
  • A song rebuilt from its folder counts as split into Lead + Backing only when both files are there. A kill between the two renames used to bring back a lead vocal without its backing; the lone file stays on disk and the next split replaces it.

Why

restore() rebuilds a song from its folder when the registry says the job was still running, or has no entry for it. It took any folder holding one stem for a finished song. collect() moves the stems in one at a time, and original.wav, mix.wav and the MP3 storage come after it, so a kill at any of those steps returned an incomplete song as "done". #898 made each file arrive whole; this makes the set of files whole.

How

  • The last thing a separation writes is stems/.complete, after every stem file and the folder are fsynced (registry.mark_stems_complete, called at the end of both pipelines). The marker goes through the existing atomic writer: fsynced temp file, rename, folder fsync.
  • _rebuild_from_folder rebuilds a folder as done only when it has the marker.
  • Songs made before this change: on startup, every song the registry holds as done gets the marker if its folder has none (only the marker is written, the stems are left alone). A song whose registry record was already gone has metadata.json, which StemDeck writes once a song is done (and now only after the marker), and is rebuilt exactly as before. Nothing is hidden, moved or deleted. Songs made by 0.20.0 after a downgrade get the marker on the way back, and 0.20.0 ignores the file.
  • If the marker cannot be written (a full disk), the job still finishes and the error is logged: failing the job would delete stems that are whole, and the song then rebuilds by its metadata.json like an older song.
  • Since the marker is the last write, a kill during the lyrics or section analysis after separation now runs the job again, where the old code kept the stems. That is the cost of "never done unless finished".

Verified

  • tests/test_stem_completeness.py runs the real pipeline steps that write stems/ (collect, original.wav, mix.wav, MP3 storage) and kills the process at eight points: one stem moved, before original.wav, before mix.wav, after mix.wav, before stem storage, one MP3 moved in, one WAV deleted, after stem storage. Each is checked with the registry kept (job queued again) and lost (the upload comes back stopped with its source). On the old code every one of these came back "done", and the split case listed a lone lead vocal. The finished run is rebuilt as done both ways, and a song from before the marker gets it from the registry.
  • A real separation on a throwaway server (MP3 storage on): stems/.complete is there, and after deleting registry.json a restart brings the song back done with every stem and the mix.
  • Marker cost, measured on this machine's SSD: 0.14 s for eight 40 MB stems written just before it, 0.004 s when they are already on disk. Not measured on a hard disk or a network share.
  • Tests whose folders stand for a finished song made by this version now carry the marker. ruff check and ruff format pass. Full Python suite: 3819 passed, 12 skipped; after the rebase onto fix(library): open an old song at once and measure its loudness in the background #899 and fix: sweep crash leftovers at startup, free a cancelled zip, give each click render its own temp #900 the 247 tests of the affected files pass again.

Does not touch uv.lock or the desktop shell.

restore() rebuilt a "done" song from any folder holding one stem, so a
process killed during collect(), before original.wav or mix.wav, or in
the middle of MP3 storage came back as a finished song with files
missing. A kill between the two vocal-split renames brought back a lead
vocal without its backing.

The last write of a separation is now stems/.complete, after every stem
file and the folder are fsynced, and recovery rebuilds a folder as done
only when it has the marker. A folder without it goes back in the queue,
or comes back as a stopped upload, and its files are kept. Songs from
before the marker get it at startup while the registry holds them as
done; one whose record was already lost is still rebuilt by its
metadata.json. A rebuilt song counts as split only when both halves of
the pair are there.
@thcp
thcp merged commit 8bac425 into big-release Nov 3, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant