Latency benchmarks for streaming speech-to-text (TTLB and P99)

Streaming speech-to-text latency should be assessed using tail metrics such as P99 and Time to Last Byte (TTLB), rather than averages, because occasional multi-second delays can disrupt conversational turn-taking even when mean latency appears low. TTLB measures the time from sending an utterance’s last audio packet to receiving its final stabilized transcript, encompassing network transit, buffering, model inference, and voice activity detection endpointing; related metrics such as Time to First Byte and word emission latency measure perceived responsiveness and live transcription quality. Production latency is affected by noisy audio, accents, language switching, packet loss, geographic distance, concurrent load, and cold starts, so the piece recommends testing at least 100 representative utterances across regions and tracking P50 through P99 distributions with pipeline-level timestamps. It argues that WebSocket connections are preferable to REST for continuous audio streaming because of lower per-chunk overhead, and that service-level agreements should commit to P99 targets rather than average latency. The publisher presents Solaria-1 as delivering first partial transcripts in under 103 milliseconds and final transcripts around 300 milliseconds, while encouraging independent validation through an open benchmark methodology and comparisons on an organization’s own real-world audio.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.