We Blind-Tested Our TTS Against Every Major Competitor. Here Are the Results.

Fish Audio conducted a comprehensive 10-day blind A/B test comparing its TTS models, S2 Pro and S1, against competitors like ElevenLabs, Inworld, and MiniMax, collecting over 5,000 preference pairs from real users who were unaware of the audio source. The results showed Fish Audio S2 Pro ranked highest with a Bradley-Terry score of 3.07, outperforming all competitors, with particularly strong performance in Chinese and Japanese languages. The study highlighted flaws in traditional evaluation metrics like MOS and WER/CER, instead using a reliable internal system based on actual user preferences, demonstrated by download behavior. Fish Audio's methodology aimed to provide a more rigorous and realistic assessment of TTS quality, accounting for real-world complexities like long-form content and multilingual support, while addressing issues with public leaderboards and platform familiarity bias. Despite spending over $2,000 on third-party APIs, Fish Audio S2 Pro emerged as the preferred model across diverse languages, validating their development approach and commitment to transparency in the TTS community.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.