Skip to content

fix(stt): chunked transcription with progress, retry, and no 300s cliff - #232

Open
EtienneLescot wants to merge 2 commits into
release/v1.8.0from
fix/stt-chunked-transcription
Open

fix(stt): chunked transcription with progress, retry, and no 300s cliff#232
EtienneLescot wants to merge 2 commits into
release/v1.8.0from
fix/stt-chunked-transcription

Conversation

@EtienneLescot

@EtienneLescot EtienneLescot commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

The bug

Transcribing a 30-minute import fails. Reproduced on the reporter's file (podcast project, 32 min):

wav 62430920 bytes
FETCH THREW after 311s: TypeError fetch failed — Headers Timeout Error

whisperServer.runMultipartInfer uses Node's global fetch, i.e. undici, whose headersTimeout defaults to 300 s. whisper.cpp sends no response header until it has transcribed the whole upload. Measured on this file: whisper needed 574 s, undici cut the connection at 300 s, and the renderer surfaced an unactionable "Transcription failed / fetch failed". The helper process kept burning CPU on a transcription nobody was listening for — the reporter's machine still had one alive a day later.

Audio extraction was never the problem: replayed as-is in Chromium it completes in 75 s and yields 31 215 421 samples @16 kHz with real signal (peak 0.85).

The fix

Split the audio into ~90 s chunks and run them one at a time.

  • electron/stt/chunking.ts nudges each boundary to the quietest 20 ms frame within ±3 s of the ideal cut, so a break lands in a pause instead of mid-word. Energy minimum, not a VAD: whisper.cpp's Silero VAD runs per request, so it cannot tell us where to cut before we upload.
  • SttManager.transcribe shifts each chunk's timestamps to absolute time, emits completedSec/totalSec per chunk, and retries a chunk up to 3 times — re-running server.start() between attempts (idempotent when the helper is alive, a respawn when it isn't), because the usual cause of a mid-run failure is a dead helper, not a bad chunk.
  • Language is pinned to whatever the first chunk detected. Left to auto-detect, whisper can flip mid-recording on a chunk that opens with a proper noun and "transcribe" the remainder as another language.
  • whisperServer bounds each request at 280 s and names the failure with the helper's stderr, instead of letting undici's 300 s ceiling surface as a bare "fetch failed".

Progress reaches the UI

The renderer's status callback forwarded only a phase string and never subscribed to the main process's events, so the toast showed one static "transcribing" for the entire run. Progress now travels the road the phase already had — status.ts owns the vocabulary, the store owns the queue, TranscriptionStatus.tsx owns how a job reads on screen. The store's onStatus also loses its as TranscriptionPhase cast: the two vocabularies are now genuinely the same type, "loading-model" included.

Both the bar and the percentage render nothing until the run reports measurable progress — queued, extracting audio and downloading the model have no fraction, and a bar pinned at 0 % reads as "stuck" where a spinner reads as "working".

Why chunks run sequentially

Measured, not assumed. whisper-stt-server holds a single model context:

wall time
two 120 s chunks, one after the other 76.9 s
the same two, fired together 144.1 s

0.53× — concurrency is ~1.9× slower. A client-side worker pool would be a pessimisation. Real parallelism needs several server processes, each with its own copy of the model resident on the GPU; worth revisiting only if a much smaller model ever becomes the default. This is recorded in the code so nobody "optimises" it later.

Verification

End to end on the 32-minute source, real planChunks driving the real server:

audio 1951.0s → 22 chunks (91,88,92,89,91,92,93,89,88,90,89,90,92,91,93,87,87,89,88,88,92,61s)
segments=485
coverage: last segment ends at 1950.7s / 1951.0s
slowest chunk 76.8s vs the 280s per-request ceiling
timestamps monotonic across chunk seams: true

Boundaries land at 87–93 s rather than on a 90 s grid — the pause search is doing its job.

  • vitest run electron/stt src/lib/captioning src/lib/ai-edition/transcription — 85/85 pass (10 new: 5 on chunk planning, 5 on orchestration — absolute offsets across seams, monotonic progress ending at 100 %, language pinning, retry, terminal failure; plus 2 on progress propagation).
  • tsc --noEmit clean.

Known gaps

  • src/lib/ai-edition/store/transcriptionStore.test.ts was not executed locally. It fails to load because i18next is missing from this machine's node_modules and no package manager is available here to install it — a pre-existing environment gap that also breaks 7 untouched component test files on this branch. The store change is covered by tsc and by the pure status.ts tests; CI should be the judge.
  • The 280 s per-request bound leaves one real ceiling: a machine so slow that a 90 s chunk needs more than 280 s (~0.3× realtime). Removing it for good means a direct undici dependency and new Agent({ headersTimeout: 0, bodyTimeout: 0 }) as the fetch dispatcher (verified working). Deliberately not done here to avoid adding a dependency for a hypothetical machine — flagged in a ponytail: comment.
  • Audio extraction still blocks the renderer for ~70 s on a 32-minute file (decodeAudioData 44 s + mixToMono 26 s, both synchronous). Out of scope here, worth its own pass.

A 30-minute recording was one `/inference` call: ~10 minutes with no
progress, no recovery from a transient failure, and — the reported bug —
killed outright before it ever finished. Node's global fetch (undici)
applies a 300s `headersTimeout`, and whisper sends no response header
until the whole upload is transcribed. Measured on the reporter's file:
whisper needed 574s, undici cut the connection at 300s, and the renderer
surfaced an unactionable "Transcription failed / fetch failed". The
helper kept burning CPU on a transcription nobody was listening for.

Split the audio into ~90s chunks and run them one at a time:

- `chunking.ts` nudges each boundary to the quietest 20ms frame within
  ±3s of the ideal cut, so a chunk break lands in a pause instead of
  mid-word. Energy minimum, not a VAD: whisper.cpp's Silero VAD runs per
  REQUEST, so it cannot tell us where to cut before we upload.
- `SttManager.transcribe` shifts each chunk's timestamps to absolute
  time, emits `completedSec`/`totalSec` per chunk, retries a chunk up to
  3 times, and re-runs `server.start()` between attempts (idempotent when
  the helper is alive, a respawn when it isn't) — the usual cause of a
  mid-run failure is a dead helper, not a bad chunk.
- The language detected on the first chunk is pinned for the rest. Left
  to auto-detect, whisper can flip mid-recording on a chunk that opens
  with a proper noun and "transcribe" the remainder as another language.
- `whisperServer` bounds each request at 280s and names the failure with
  the helper's stderr, instead of letting undici's 300s ceiling surface
  as a bare "fetch failed".

Chunks run SEQUENTIALLY, measured rather than assumed: whisper-stt-server
holds a single model context, and two 120s chunks took 76.9s one after
the other vs 144.1s fired together (0.53x — concurrency is ~1.9x SLOWER).
A client-side worker pool would be a pessimisation.

The progress reaches the UI: the renderer's status callback forwarded
only a phase string and never subscribed to the main process's events, so
the toast showed one static "transcribing" for the whole run. It now
carries the chunk progress and renders a real bar.

Verified end to end on a 32-minute source: 22 chunks, timestamps
monotonic across every seam, last segment at 1950.7s of 1951.0s, slowest
chunk 76.8s against the 280s ceiling.
…nner

The chunked pipeline now reports how much audio it has transcribed, so
surface it. A 30-minute recording spends minutes in "Transcribing…", and
a spinner that never changes is indistinguishable from a hang.

Progress travels the same road the phase already did — `status.ts` owns
the vocabulary, the store owns the queue, `TranscriptionStatus.tsx` owns
how a job reads on screen:

- `TranscriptionProgress` + `progressFraction` join `TranscriptionPhase`
  in status.ts, and `deriveAssetStatus` carries them onto the view.
- The store's `onStatus` no longer casts its argument to
  `TranscriptionPhase`: the renderer's `TranscribeStatus` and
  `TranscriptionPhase` are now genuinely the same vocabulary, including
  the `"loading-model"` phase the cast used to paper over. Failure paths
  clear `progress` with `phase`, so a failed job cannot leave a stale bar.
- `TranscriptionStatusDot`'s sibling `TranscriptionProgressBar` renders
  a determinate bar, and the label gains a percentage.

Both render nothing until the run reports measurable progress. Queued,
extracting audio and downloading the model have no fraction to report,
and a bar pinned at 0% reads as "stuck" where the spinner reads as
"working".
@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 686152ad-aac1-4a1a-9e62-e2791b4faa1a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant