fix(stt): chunked transcription with progress, retry, and no 300s cliff - #232
Open
EtienneLescot wants to merge 2 commits into
Open
fix(stt): chunked transcription with progress, retry, and no 300s cliff#232EtienneLescot wants to merge 2 commits into
EtienneLescot wants to merge 2 commits into
Conversation
A 30-minute recording was one `/inference` call: ~10 minutes with no progress, no recovery from a transient failure, and — the reported bug — killed outright before it ever finished. Node's global fetch (undici) applies a 300s `headersTimeout`, and whisper sends no response header until the whole upload is transcribed. Measured on the reporter's file: whisper needed 574s, undici cut the connection at 300s, and the renderer surfaced an unactionable "Transcription failed / fetch failed". The helper kept burning CPU on a transcription nobody was listening for. Split the audio into ~90s chunks and run them one at a time: - `chunking.ts` nudges each boundary to the quietest 20ms frame within ±3s of the ideal cut, so a chunk break lands in a pause instead of mid-word. Energy minimum, not a VAD: whisper.cpp's Silero VAD runs per REQUEST, so it cannot tell us where to cut before we upload. - `SttManager.transcribe` shifts each chunk's timestamps to absolute time, emits `completedSec`/`totalSec` per chunk, retries a chunk up to 3 times, and re-runs `server.start()` between attempts (idempotent when the helper is alive, a respawn when it isn't) — the usual cause of a mid-run failure is a dead helper, not a bad chunk. - The language detected on the first chunk is pinned for the rest. Left to auto-detect, whisper can flip mid-recording on a chunk that opens with a proper noun and "transcribe" the remainder as another language. - `whisperServer` bounds each request at 280s and names the failure with the helper's stderr, instead of letting undici's 300s ceiling surface as a bare "fetch failed". Chunks run SEQUENTIALLY, measured rather than assumed: whisper-stt-server holds a single model context, and two 120s chunks took 76.9s one after the other vs 144.1s fired together (0.53x — concurrency is ~1.9x SLOWER). A client-side worker pool would be a pessimisation. The progress reaches the UI: the renderer's status callback forwarded only a phase string and never subscribed to the main process's events, so the toast showed one static "transcribing" for the whole run. It now carries the chunk progress and renders a real bar. Verified end to end on a 32-minute source: 22 chunks, timestamps monotonic across every seam, last segment at 1950.7s of 1951.0s, slowest chunk 76.8s against the 280s ceiling.
…nner The chunked pipeline now reports how much audio it has transcribed, so surface it. A 30-minute recording spends minutes in "Transcribing…", and a spinner that never changes is indistinguishable from a hang. Progress travels the same road the phase already did — `status.ts` owns the vocabulary, the store owns the queue, `TranscriptionStatus.tsx` owns how a job reads on screen: - `TranscriptionProgress` + `progressFraction` join `TranscriptionPhase` in status.ts, and `deriveAssetStatus` carries them onto the view. - The store's `onStatus` no longer casts its argument to `TranscriptionPhase`: the renderer's `TranscribeStatus` and `TranscriptionPhase` are now genuinely the same vocabulary, including the `"loading-model"` phase the cast used to paper over. Failure paths clear `progress` with `phase`, so a failed job cannot leave a stale bar. - `TranscriptionStatusDot`'s sibling `TranscriptionProgressBar` renders a determinate bar, and the label gains a percentage. Both render nothing until the run reports measurable progress. Queued, extracting audio and downloading the model have no fraction to report, and a bar pinned at 0% reads as "stuck" where the spinner reads as "working".
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
Transcribing a 30-minute import fails. Reproduced on the reporter's file (
podcastproject, 32 min):whisperServer.runMultipartInferuses Node's globalfetch, i.e. undici, whoseheadersTimeoutdefaults to 300 s. whisper.cpp sends no response header until it has transcribed the whole upload. Measured on this file: whisper needed 574 s, undici cut the connection at 300 s, and the renderer surfaced an unactionable "Transcription failed / fetch failed". The helper process kept burning CPU on a transcription nobody was listening for — the reporter's machine still had one alive a day later.Audio extraction was never the problem: replayed as-is in Chromium it completes in 75 s and yields 31 215 421 samples @16 kHz with real signal (peak 0.85).
The fix
Split the audio into ~90 s chunks and run them one at a time.
electron/stt/chunking.tsnudges each boundary to the quietest 20 ms frame within ±3 s of the ideal cut, so a break lands in a pause instead of mid-word. Energy minimum, not a VAD: whisper.cpp's Silero VAD runs per request, so it cannot tell us where to cut before we upload.SttManager.transcribeshifts each chunk's timestamps to absolute time, emitscompletedSec/totalSecper chunk, and retries a chunk up to 3 times — re-runningserver.start()between attempts (idempotent when the helper is alive, a respawn when it isn't), because the usual cause of a mid-run failure is a dead helper, not a bad chunk.whisperServerbounds each request at 280 s and names the failure with the helper's stderr, instead of letting undici's 300 s ceiling surface as a bare "fetch failed".Progress reaches the UI
The renderer's status callback forwarded only a phase string and never subscribed to the main process's events, so the toast showed one static "transcribing" for the entire run. Progress now travels the road the phase already had —
status.tsowns the vocabulary, the store owns the queue,TranscriptionStatus.tsxowns how a job reads on screen. The store'sonStatusalso loses itsas TranscriptionPhasecast: the two vocabularies are now genuinely the same type,"loading-model"included.Both the bar and the percentage render nothing until the run reports measurable progress — queued, extracting audio and downloading the model have no fraction, and a bar pinned at 0 % reads as "stuck" where a spinner reads as "working".
Why chunks run sequentially
Measured, not assumed. whisper-stt-server holds a single model context:
0.53× — concurrency is ~1.9× slower. A client-side worker pool would be a pessimisation. Real parallelism needs several server processes, each with its own copy of the model resident on the GPU; worth revisiting only if a much smaller model ever becomes the default. This is recorded in the code so nobody "optimises" it later.
Verification
End to end on the 32-minute source, real
planChunksdriving the real server:Boundaries land at 87–93 s rather than on a 90 s grid — the pause search is doing its job.
vitest run electron/stt src/lib/captioning src/lib/ai-edition/transcription— 85/85 pass (10 new: 5 on chunk planning, 5 on orchestration — absolute offsets across seams, monotonic progress ending at 100 %, language pinning, retry, terminal failure; plus 2 on progress propagation).tsc --noEmitclean.Known gaps
src/lib/ai-edition/store/transcriptionStore.test.tswas not executed locally. It fails to load becausei18nextis missing from this machine'snode_modulesand no package manager is available here to install it — a pre-existing environment gap that also breaks 7 untouched component test files on this branch. The store change is covered bytscand by the purestatus.tstests; CI should be the judge.undicidependency andnew Agent({ headersTimeout: 0, bodyTimeout: 0 })as the fetch dispatcher (verified working). Deliberately not done here to avoid adding a dependency for a hypothetical machine — flagged in aponytail:comment.decodeAudioData44 s +mixToMono26 s, both synchronous). Out of scope here, worth its own pass.