Open Source AI Benchmarking toolkit for benchmarking speech to text services
-
Updated
Apr 17, 2024 - Python
#
stt-benchmark
Here are 8 public repositories matching this topic...
🐍📦 Ultra-fast Python package for calculating and analyzing the Word Error Rate (WER). Built for the scalable evaluation of speech and transcription accuracy.
python nlp metrics levenshtein-distance automatic-speech-recognition speech-to-text stt asr python-package wer word-error-rate stt-benchmark asr-evaluation werpy
-
Updated
Oct 8, 2026 - Python
Phone and word timestamp accuracy on human-labeled TIMIT and Buckeye, clean and under reverb, noise, music and babble. Two tracks, forced aligners given the transcript and ASR systems that find their own words. MFA, WhisperX, Charsiu, Qwen3, Parakeet-TDT, Olign and more. Monthly leaderboards, paper on arXiv.
benchmark timestamp stt asr forced-alignment boundary asr-benchmark asr-model stt-benchmark asr-services fa-benchmark
-
Updated
Oct 12, 2026 - Python
Core engine behind Calibrate, a framework for evaluating AI agents: speech-to-text, text-to-speech, LLM evaluation, end-to-end simulations
ai-agents stt-benchmark voice-ai asr-evaluation ai-engineering ai-evaluation pipecat llm-evaluation llm-evaluation-toolkit llm-evaluation-framework voice-agents llm-evals ai-agent-evaluation agent-evaluation ai-evaluation-framework llm-benchmark ai-evals agent-evaluation-tools tts-evaluation tts-benchmark
-
Updated
Oct 12, 2026 - JavaScript
Backend for Calibrate, a framework for evaluating AI agents: speech-to-text, text-to-speech, LLM evaluation, end-to-end simulations
ai-agents stt-benchmark voice-ai asr-evaluation ai-engineering ai-evaluation pipecat llm-evaluation llm-evaluation-framework voice-agents ai-evaluation-tools llm-evals ai-evaluation-framework llm-benchmark ai-evals voice-agent-evals tts-evaluation
-
Updated
Oct 12, 2026 - Python
Frontend for Calibrate, a framework for evaluating AI agents: speech-to-text, text-to-speech, LLM evaluation, end-to-end simulations
ai-agents stt-benchmark voice-ai asr-evaluation ai-evaluation pipecat llm-evaluation llm-evaluation-toolkit voice-agents llm-evals agent-evals ai-agent-evaluation agent-evaluation ai-evaluation-framework llm-benchmark ai-evals voice-agent-evals agent-evaluation-tools tts-evaluation tts-benchmark
-
Updated
Oct 12, 2026 - TypeScript
Run AI agent evaluations in your CI/CD pipeline using Calibrate and catch regressions
ci-cd speech-to-text github-action stt-benchmark ai-evaluation llm-evaluation llm-evaluation-toolkit llm-evaluation-framework voice-agents ai-evaluation-tools agent-evals ai-agent-evaluation agent-evaluation llm-benchmark ai-evals voice-agent-evals agent-evaluation-tools
-
Updated
Aug 15, 2026 - Shell
Python SDK for Calibrate's API
stt-benchmark voice-ai ai-engineering ai-evaluation llm-evaluation llm-evaluation-toolkit llm-evaluation-framework voice-agents ai-evaluation-tools llm-evals agent-evals agent-evaluation llm-benchmark ai-evals voice-agent-evals agent-evaluation-tools tts-benchmark
-
Updated
Oct 12, 2026 - Python
Add this topic to your repo
To associate your repository with the stt-benchmark topic, visit your repo's landing page and select "manage topics."