Adding Benchmaxxer Repellant to the Open ASR Leaderboard

The Open ASR Leaderboard, launched in September 2023, has become a significant platform for benchmarking speech recognition models, attracting over 710,000 visits. The platform aims to enhance transparency and standardization in evaluating ASR models by integrating public datasets and recently curated high-quality private datasets from Appen Inc. and DataoceanAI, which cover diverse accents and speech styles. These private datasets are kept confidential to prevent "benchmaxxing," where models might artificially boost performance without real-world robustness. The leaderboard maintains openness by open-sourcing its UI code and evaluation scripts, which allows for community contributions and improvements. Standardization is achieved by normalizing model outputs and transcripts, ensuring consistency in punctuation and casing. The leaderboard's objectives include capturing the nuanced performance differences across models tailored for various accents, languages, and use cases, and it provides options to customize evaluations with private and public datasets to reflect specific application needs. The platform continues to evolve to include evaluations under noisy real-world conditions and is open to feedback and contributions to further refine its benchmarking capabilities.