How does Kaldi ASR compare with Mozilla DeepSpeech in terms of the speech recognition accuracy (e.g., in terms of word error rate)?


https://web.archive.org/web/20171130080859/https://hacks.mozilla.org/2017/11/a-journey-to-10-word-error-rate/ :

Our word error rate on LibriSpeech’s test-clean set is 6.5%, which not only achieves our initial goal, but gets us close to human level performance. [...] (5.83% according to the Deep Speech 2 paper). On a MacBook Pro, using the GPU, the model can do inference at a real-time factor of around 0.3x, and around 1.4x on the CPU alone. (A real-time factor of 1x means you can transcribe 1 second of audio in 1 second.)

Bonus: Facebook AI Research Automatic Speech Recognition Toolkit (Torch+lua, BSD License) gets 4.8% WER test-clean and 14.5% WER test-other on the LibriSpeech corpus.

  • Since you found the answer, you could mark your own answer as "accepted". Otherwise, the question will keep getting bumped to the front page by SE bots. – prash Dec 30 '17 at 12:42
  • @prash Good point, done – Franck Dernoncourt Jan 1 '18 at 8:26
  • WER is not the only parameter we should be measuring how one ASR library fares against the other, a few other parameters can be: how good they fare in noisy scenarios, how easy is it to add vocabulary, what is the real-time factor, how robustly the trained model responds to changes in accent intonation etc. – absin Feb 19 '19 at 4:03
  • @AbSin 100% agreed. – Franck Dernoncourt Feb 19 '19 at 4:04

Kaldi provides WER of 4.28% whereas deepspeech gives 5.83% on librispeech clean data. Check this out: https://github.com/syhw/wer_are_we

  • 2
    Thanks, it's not Mozilla DeepSpeech though, but Baidu's :/ – Franck Dernoncourt Nov 23 '17 at 17:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.