Skip to main content
Linguistics

Return to Answer

edited body
Source Link
phipsgabler
phipsgabler
  • 508
  • 3
  • 14

I now took it upon myself to just redraw the figure and update it with numbers collected on https://github.com/syhw/wer_are_we:

Redrawn benchmark historyRedrawn benchmark history

(SVG version here). Use with care, the y positions of the new data points are more or less eyeballed.

More data in the form of tables and figures can be found in

  • P. Szymański et al., “WER we are and WER we think we are,” in Findings of the Association for Computational Linguistics: EMNLP 2020, Online, Nov. 2020, pp. 3290–3295. doi: 10.18653/v1/2020.findings-emnlp.295.
  • T. Likhomanenko et al., “Rethinking Evaluation in ASR: Are Our Models Robust Enough?” arXiv, May 02, 2021. doi: 10.48550/arXiv.2010.11745.

I now took it upon myself to just redraw the figure and update it with numbers collected on https://github.com/syhw/wer_are_we:

Redrawn benchmark history

(SVG version here). Use with care, the y positions of the new data points are more or less eyeballed.

More data in the form of tables and figures can be found in

  • P. Szymański et al., “WER we are and WER we think we are,” in Findings of the Association for Computational Linguistics: EMNLP 2020, Online, Nov. 2020, pp. 3290–3295. doi: 10.18653/v1/2020.findings-emnlp.295.
  • T. Likhomanenko et al., “Rethinking Evaluation in ASR: Are Our Models Robust Enough?” arXiv, May 02, 2021. doi: 10.48550/arXiv.2010.11745.

I now took it upon myself to just redraw the figure and update it with numbers collected on https://github.com/syhw/wer_are_we:

Redrawn benchmark history

(SVG version here). Use with care, the y positions of the new data points are more or less eyeballed.

More data in the form of tables and figures can be found in

  • P. Szymański et al., “WER we are and WER we think we are,” in Findings of the Association for Computational Linguistics: EMNLP 2020, Online, Nov. 2020, pp. 3290–3295. doi: 10.18653/v1/2020.findings-emnlp.295.
  • T. Likhomanenko et al., “Rethinking Evaluation in ASR: Are Our Models Robust Enough?” arXiv, May 02, 2021. doi: 10.48550/arXiv.2010.11745.
Source Link
phipsgabler
phipsgabler
  • 508
  • 3
  • 14

I now took it upon myself to just redraw the figure and update it with numbers collected on https://github.com/syhw/wer_are_we:

Redrawn benchmark history

(SVG version here). Use with care, the y positions of the new data points are more or less eyeballed.

More data in the form of tables and figures can be found in

  • P. Szymański et al., “WER we are and WER we think we are,” in Findings of the Association for Computational Linguistics: EMNLP 2020, Online, Nov. 2020, pp. 3290–3295. doi: 10.18653/v1/2020.findings-emnlp.295.
  • T. Likhomanenko et al., “Rethinking Evaluation in ASR: Are Our Models Robust Enough?” arXiv, May 02, 2021. doi: 10.48550/arXiv.2010.11745.