Up-to-date performance comparison of speech recognition over time

Question

I am looking for an up-to-date (as of 2022) figure or table which displays the improvement in speech recognition on benchmarks for read and spontaneous speech over time, ideally in terms of word error rate. Somehing like this graph (from [1], reprinted in [2]):

but including more recent developments after 2009, especially results involving deep learning. The latest I could find (via inverse image search) is an Economist article from 2017, which does however not a good job of comparison (and their "sources" are, unfortunately, rather ridiculous: "Microsoft, research papers").

---

[1] J. Ajot and J. Fiscus, "Speech-To-Text (STT) and Speaker Attributed STT (SASTT) Results", NIST Rich Transcription Evaluation Workshop, 2009.

[2] T. Valenta and L. Šmídl, “Word confusions in the transcription and recognition of spontaneous Czech,” in Tackling the Complexity in Speech, vol. XIV, O. Niebuhr and R. Skarnitzl, Eds. Prague: Faculty of Arts, Charles University, 2015.

Answer 1

I now took it upon myself to just redraw the figure and update it with numbers collected on https://github.com/syhw/wer_are_we:

(SVG version here). Use with care, the y positions of the new data points are more or less eyeballed.

More data in the form of tables and figures can be found in

• P. Szymański et al., “WER we are and WER we think we are,” in Findings of the Association for Computational Linguistics: EMNLP 2020, Online, Nov. 2020, pp. 3290–3295. doi: 10.18653/v1/2020.findings-emnlp.295.
• T. Likhomanenko et al., “Rethinking Evaluation in ASR: Are Our Models Robust Enough?” arXiv, May 02, 2021. doi: 10.48550/arXiv.2010.11745.