ArmBench-ASR: A Benchmark for Armenian ASR

ArmBench-ASR v0.1 is a benchmark designed to make Armenian automatic speech recognition systems easier to compare by evaluating nearly 30 open-weight and closed models across 10,113 audio clips, totaling about 20.7 hours, from five datasets representing read speech, poetry, movies, and narrated news. It reports both strict and normalized word and character error rates using consistent preprocessing, with Gemini 2.5 Pro achieving the best combined strict WER of 14.31%, while NVIDIA’s Armenian FastConformer is the highest-ranking open model at 20.21%. Results vary substantially by domain, with movie dialogue proving the most difficult category and normalization showing that punctuation, capitalization, and orthographic variation contribute significantly to measured errors. Closed systems occupy the eight best aggregate positions, although Armenian open models lead on Common Voice and HiSpeech models perform strongly on poetry. The benchmark is primarily focused on Eastern Armenian, includes three private datasets that limit full reproducibility, and measures transcription accuracy rather than features such as diarization, timestamps, or long-form performance. Future versions aim to add dialects, code-switching, conversational and specialized speech, speaker diarization, and broader transcription-quality evaluations.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.