Company
Date Published
Author
-
Word count
1990
Language
English
Hacker News points
None

Summary

We recently added a new open test set to our evaluation suite which covers many languages and ran the numbers to compare ourselves against some others using publicly available data called the FLEURS dataset. We performed really well in underrepresented languages, outperforming our competitors by significant margins. We were also more accurate than Amazon, AssemblyAI, and Deepgram across all languages we offer. Our accuracy rate was 93.73% of the time when compared to every major ASR vendor. However, we recognize that these results are not entirely representative of real-world scenarios, as they are based on artificially clean data. To address this, we've made our test data more realistic by incorporating varied audio from different speakers, accents, and environments. Our tests show that Speechmatics is making significant improvements over OpenAI Whisper in these more challenging scenarios. We believe that accuracy is not the only metric to judge a provider, but rather usefulness in the real world. Our commitment is to achieve high accuracy regardless of input quality, even when it's low or noisy. To truly test this, we encourage users to try out our portal with their own audio and see the results for themselves.