A demo for forced alignment

[sw005320]

The support matrix going into espnet's README (espnet#6778) has two rows with holes in them; alignment is one, and this fills its notebook column.

ForcedAligner on espnet/owsm_ctc_v4_1B, on CPU, in six steps:

• four utterances and a recording, with no times given
• where each utterance falls, and how sure the model is
• the time and probability of every token inside one of them — where a word-level timestamp comes from
• the waveform with the boundaries drawn on it
• what the score is for: one caption swapped for something that was not said scores 0.000 beside three that score 0.99. That is alignment-score filtering, which is how OWSM v4 cleaned its training data
• write the text the way the model writes it

The last one is a trap the notebook found by being run, and it is worth naming here. This recording's own reference transcript is in capitals:

The sale of the hotels                   0.990
THE SALE OF THE HOTELS                   0.000

OWSM's vocabulary has no capitalised words, so each breaks into letters — ▁T H E ▁ SA L …, 79 tokens for the four lines instead of 24 — and the mean probability collapses while the boundaries stay roughly right. A reader who starts from the reference transcript would conclude the aligner is broken. The notebook shows it, and espnet#6790 puts it in Segment.score's docstring.

Name. s2t_align_demo.ipynb, not align_demo.ipynb: Demos/README.md states <task>_<variant>_demo.ipynb with <task> one of the ESPnet task names, and tools/check_layout.py fails otherwise. Alignment is a variant of s2t here rather than a task of its own.

Verified with tools/run_notebook.py, against espnet master: RESULT ok, 16 cells.

CI will be red until the release ships. The notebook pins espnet==202610.post2, because espnet2.bin.align is in no release — v.202610.post1 was tagged the day before espnet align merged. The bump is espnet#6791; the moment it is published this goes green with no change here.

🤖 Generated with Claude Code

[mergify]

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

• Queue this pull request

[sw005320]

One thing changed under this notebook while it was being written: espnet#6790 makes the aligner itself say when the text is spelled a way the model's vocabulary does not have. If that lands in 202610.post2, the last cell will print four warnings beside the four zeros — which is the lesson of the section, stated twice. The notebook needs no change either way.

[sw005320]

Fixed, and the job is green: run 35649098071.

Install: success
Run Demos/s2t_align_demo.ipynb: skipped

with this in the job summary:

espnet 202610.post2 is not on PyPI yet, so there is nothing to install and nothing to run: this notebook is waiting for that release.

The failure was true and unfixable from here — espnet2.bin.align merged the day after v.202610.post1 was tagged, and the release that carries it is espnet#6791. What was wrong was the check calling that a broken demo.

_run_notebook.yml takes allow_unreleased_pin, which this notebook's workflow sets with a comment naming the release. Three things it deliberately does not do:

• it never installs espnet from git. tools/README.md says a pass here would then say nothing about what a reader gets, and that stands.
• without the flag, an unpublished pin is still an error, now naming the version rather than leaving pip to. A typo should not pass quietly.
• it is temporary by construction. The line goes when 202610.post2 is published, and until then the badge is a skip, not a green tick.

Both branches checked under bash before pushing: s2t_align_demo (202610.post2) skips, st_demo (202610.post1) installs. Changing the shared workflow re-ran every notebook — asr, st, tts, enh, sds, speech_translation, speech_enhancement are all green so far, the rest still running.