Skip to content

About

No description, website, or topics provided.

Resources

Stars

1 star

Watchers

0 watching

Forks

Latest commit

 

History

10 Commits

Folders and files

Repository files navigation

olign-client

Examples, test data and tests for calling olign, Olewave's forced alignment and pronunciation scoring for English speech. You send audio (and, optionally, the text that was said); olign returns every word and phone with its start and end time and a 0–100 score.

Which version?

You want to… Use Endpoint Access
reproduce olign's results in the fa-bench paper v0.9 https://api.olewave.com/olign/v0.9 a service token from Olewave (info@olewave.com)
get the latest olign results for your own recordings, or score Olign 1.0 with fa-bench v1.0 https://api.olewave.com/olign/v1 an API key, self-service with an invitation code
  • v0.9 serves Olign 0.9, the version behind every olign number in the fa-bench paper: one clip and its text per request, answered in the same call. Since 2026-10-11 it serves Olign 0.9.1; see Reproduce the fa-bench numbers. The paper's first arXiv version calls it "Olign 1.0" in a table caption; the revised paper and fa-bench's records name it Olign 0.9.
  • v1.0 serves Olign 1.0, the current API: upload a whole recording (the transcript is optional), then poll the job for word and phone timings.

v1.0: whole recordings, as jobs

Set up

  1. Get an invitation code from Olewave (info@olewave.com).

  2. Sign up at https://tycho.olewave.com/#signup with your email, a password and the code.

  3. Create an API key under Profile → API keys, named after the server that will use it. It is shown once and it identifies you, so keep it out of code you share. You can hold several and revoke each on its own:

    read -rsp 'olign key: ' OLIGN_KEY && export OLIGN_KEY && echo
  4. Add a card under Billing. Saving it charges nothing and gives 3,000 free credits, 1,000 minutes of audio at the launch price; buy more there when needed.

Call it

Two calls: submit, then poll until the job is done.

# submit: audio (any common format), transcript optional  ->  202 {"job_id": "..."}
curl -H "X-API-Key: $OLIGN_KEY" \
  -F audio=@examples/short1.wav -F transcript=@examples/short1.txt \
  https://api.olewave.com/olign/v1

# poll every 5-10 s until "state" is "done"; the result is in the same response
curl -H "X-API-Key: $OLIGN_KEY" https://api.olewave.com/olign/v1/<job_id>

Or let the example do both and save the result (Python standard library only):

python3 examples/v1_client.py examples/short1.wav --transcript examples/short1.txt -o result.json

With a transcript, the job's status reports "transcript_source": "client".

Leave the transcript part out and olign makes the transcript itself, then aligns and scores what it heard:

curl -H "X-API-Key: $OLIGN_KEY" -F audio=@examples/short1.wav https://api.olewave.com/olign/v1
  • The status reports "transcript_source": "asr".
  • If it finds no speech, the job ends failed, and log_tail says no transcript given and no speech recognized in the audio.

Send a transcript when you're assessing pronunciation. Without one, words are scored against what the recognizer heard: a mispronounced word may be recognized as what was actually said and look fine. Audio-only mode is best when you mainly need timings.

Handle the result

Once state is done, the response holds:

field meaning
audio_seconds length of your audio, as decoded; your daily usage is counted on it
transcript_source client when you sent the transcript, asr when olign made it
result.words[] every transcript word in order: word, begin, end (seconds), score (0–100), sent_id, and phones[] with ph (CMU/ARPAbet), begin, end, score
result.sentence_scores[] per sentence: sent_score, word_avg, word_min
result.stats.overall_mean the mean segment score; n_err > 0 means some segments could not be scored
  • A word that was not spoken has "spoken": false and zero duration; a [bracketed] tag has "annotation": true.
  • A phone with zero duration could not be placed; it sits inside its word, where it falls in order.
  • Keep the X-Olign-*-Version response headers with any score you store (the example saves them under _versions): scores change between versions.

examples/v1_process.py turns a saved result into a summary, spreadsheets and a Praat TextGrid. examples/sample_v1_result.json is a real result for examples/short1.wav, so you can try it without an account:

python3 examples/v1_process.py examples/sample_v1_result.json --csv out/ --textgrid out/short1.TextGrid

This prints the overall score and the weakest words, and writes out/words.csv, out/phones.csv and out/short1.TextGrid, which opens in Praat as a words tier and a phones tier.

Limits and errors

credits 3.5 credits per minute of audio, 3 during the launch promotion, rounded up per job; 1,000 credits = $1, so an hour costs $0.21 ($0.18 during the promotion). A job is charged when it is accepted and refunded if it fails. Every response says what it cost and what is left: X-Credits-Charged, X-Credits-Balance
monthly limit optional, yours to set under Settings: once this month's usage reaches it, new jobs are refused (429) until the 1st (UTC)
usage per day and per API key on the dashboard's Usage page, or GET https://tycho.olewave.com/v1/usage with your key
length files up to 3 hours; a job takes about 0.06× the audio length
size 100 MB per upload. Compress long recordings, e.g. Opus at 32 kbit/s is ~14 MB an hour: ffmpeg -i in.wav -ac 1 -c:a libopus -b:a 32k out.ogg
queue several jobs run at once; yours may wait as queued
your data Olewave keeps your audio, transcript and results for your account, as the beta testing agreement says; ask info@olewave.com to delete them
User-Agent send your own: Cloudflare rejects some HTTP-library defaults (403, error code: 1010, e.g. Python's urllib)
code when
202 submitted; the body has job_id
401 missing, invalid or revoked API key
402 out of credits, or the recording is longer than your credits pay for (the message gives both lengths): add a card or buy credits under Billing
404 unknown job id, one submitted with another key, or a job still running when the service restarted: resubmit (finished jobs survive restarts)
411 no Content-Length (a chunked upload)
413 over 100 MB, or longer than 3 hours; audio_seconds in the body is the recording's length
415 not multipart/form-data
422 no part named audio
429 your monthly limit is reached, or the recording is longer than what is left of it; Retry-After gives the seconds until the 1st, 00:00 UTC. The limit is yours: raise or remove it under Settings

A job can also end with "state": "failed"; log_tail says why.

Score Olign 1.0 with fa-bench

fa-bench runs Olign 1.0 through this API, one job per utterance, in both modes:

fa-bench recipe sends fa-bench track
olign_v1 the audio and its reference transcript (transcript_source: client) 1, alignment given the words
olign_v1_asr the audio alone (transcript_source: asr): Olign 1.0 recognizes the words, then aligns its own 2, Olign's own recognizer then its aligner, never another system's words
  1. Set up fa-bench as its README describes. The corpora are licensed separately.
  2. Put your key in fa-bench's .fabench.env (untracked): OLIGN_API_KEY=<key>.
  3. Align and score: evals/run_evals.sh olign_v1 or evals/run_evals.sh olign_v1_asr; add --use-noisy-dataset for the four degraded conditions.

Every response is cached, so a restart or a rescore sends nothing again. A full sweep, every split of both corpora clean and degraded, is 47,805 jobs and 34.1 hours of audio per mode. The utterances are a few seconds each, so nearly every job rounds up to one credit: about 47,800 credits, or $48, per mode.


v0.9: one clip per request (reproducing fa-bench)

Set up

Ask Olewave (info@olewave.com) for a Cloudflare Access service token. Every request sends it as two headers; without them the answer is 403.

export CF_ACCESS_CLIENT_ID=<id>.access CF_ACCESS_CLIENT_SECRET=<secret>

Call it

POST the audio as the raw request body; everything else goes in the query string.

curl -X POST --data-binary @examples/short1.wav \
  -H "CF-Access-Client-Id: $CF_ACCESS_CLIENT_ID" \
  -H "CF-Access-Client-Secret: $CF_ACCESS_CLIENT_SECRET" \
  "https://api.olewave.com/olign/v0.9?ref_text=what%20is%20better%20a%20fruit%20or%20hamburger&core_type=en.word.score&rank=100&accent=2&show_phone_details=1"

Or run examples/rest_client.sh, which reads the two variables and prints a summary:

./examples/rest_client.sh --url https://api.olewave.com/olign/v0.9 --core-type en.word.score \
  --wav examples/short1.wav --ref-text "what is better a fruit or hamburger"
query parameter default values
ref_text (required) what the speaker was supposed to say, URL-encoded
core_type en.sent.score en.word.score, en.sent.score, en.sent.child, en.pred.exam, en.phone.align
rank 100 score scale; 100 means scores are 0–100
accent 2 1 = UK, 2 = US
show_phone_details 1 1 includes the per-phone breakdown

The body is any common audio format, up to 50 MB.

Handle the result

The response is one JSON object:

{
  "versions": {"api": "v0.9.0"},
  "errId": 0,              // must be 0; otherwise read "error" and ignore "result"
  "error": "",
  "result": {
    "overall": 87,         // the headline score, 0..rank
    "wavetime": 4480,      // audio scored, in milliseconds
    "details": [           // one entry per word of ref_text, in order
      {"char": "what", "score": 55, "begin": 470, "end": 750, "dur": 280,   // milliseconds
       "phone": [{"char": "w", "score": 0, "begin": 470, "end": 513, "dur": 43}]}
    ]
  }
}
  • Check errId, not just the HTTP status: it can be non-zero on a 200.
  • All times are milliseconds: begin: 470 is 0.47 s.
  • details[] has one entry per word of ref_text, even words the speaker never said; a near-zero score means mispronounced or skipped.
  • Keep versions with any score you store. It holds the same values as the X-Olign-*-Version headers. Answers from before 2026-10-11 23:46 UTC don't have it; take the versions from their headers instead.
d = response.json()
if d["errId"] != 0:
    raise RuntimeError(d["error"])
for w in d["result"]["details"]:
    print(w["char"], w["score"], w["begin"] / 1000, w["end"] / 1000)
code when
200 parse the body, then check errId
400 audio that cannot be decoded
403 missing or invalid service token
411 no Content-Length (a chunked upload)
413 body over 50 MB
422 ref_text missing

Reproduce the fa-bench numbers

fa-bench makes exactly this call: one request per utterance with the whole audio, several in parallel.

  1. Set up fa-bench as its README describes. The corpora are licensed separately; fa-bench works from your own copies.

  2. Point its olign aligner here, in fa-bench's .fabench.env (untracked):

    OLIGN_BASE_URL=https://api.olewave.com/olign/v0.9
    CF_ACCESS_CLIENT_ID=<id>.access
    CF_ACCESS_CLIENT_SECRET=<secret>
  3. Align and score: evals/run_evals.sh olign.

Check the version first. The paper's olign numbers are Olign 0.9's, which this endpoint serves through API v0.9 (the paper's first arXiv version says "Olign 1.0" in a table caption; the revised paper and fa-bench's records name it Olign 0.9). Every response carries its versions in X-Olign-*-Version headers and, since 2026-10-11, in the body's versions.

Since 2026-10-11 this endpoint serves Olign 0.9.1. The paper's current version reports Olign 0.9.0, which is no longer served. Olign 0.9.1's word and phone times differ from 0.9.0's, so a run today won't match the table below exactly; a later version of the paper will report Olign 0.9.1. The table stays as the record of 0.9.0.

What 0.9.0 gave: the paper's alignments were made in August 2026 on a pre-release server, not on the public build, so individual boundaries differ by a few milliseconds (8.9 ms median per utterance), but the totals agree. Buckeye test (clean), scored with fa-bench's current scorer:

Buckeye test, clean paper public build 98728396 (0.9.0)
word MAE (ms) 22.5 22.3
word F1 @20 ms 0.744 0.747
phone MAE (ms) 12.7 12.3
PER (%) 28.2 28.2
phone F1 @20 ms 0.522 0.526

The public build gave identical timings for the same file on every run. Allow about 0.1 ms and 0.002 for scorer drift: the current scorer gives the paper's own alignments 22.6 ms word MAE and 0.524 phone F1. Only Buckeye test (clean) has been re-measured so far.


Tests

test needs checks
python3 -m unittest tests.test_v1_offline -v nothing examples/v1_process.py on the sample result: summary, CSVs, and TextGrid tiers with no gaps or overlaps
python3 -m unittest tests.test_v1_live -v OLIGN_KEY every clip in test-data/ through the v1.0 API: the length, that nearly every word comes back spoken, times in order and inside the audio, every phone inside its word, scores 0–100, and that v1_process.py reads the result. About a minute, and 5.5 minutes of audio in four jobs: 18 credits at the launch price, 21 after
the rest_client.sh call above the service token the v0.9 endpoint: short1.wav with en.word.score must give overall=87 words=7 wavetime=4480 (ms)

test-data/ holds three LibriSpeech recordings with their transcripts (11 s of FLAC, 56 s and 3.4 min of Opus), credited in test-data/README.md; test-data/cases.json lists what the live test sends and expects.

License

Copyright 2026 Olewave, LLC (https://www.olewave.com). The code and documentation here are licensed under the Apache License 2.0. The recordings in test-data/ are from LibriSpeech and keep their own license, CC BY 4.0; see test-data/README.md.

About

No description, website, or topics provided.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages