What does "inference is slower than realtime" mean?

LiveKit voice-agent warnings stating that VAD inference is slower than realtime indicate that the voice activity detection model running within the agent process is falling behind incoming audio, rather than a problem with LiveKit Inference STT, TTS, or LLM services. Because VAD detects when users begin and stop speaking, persistent delay can cause late responses, missed interruptions, clipped greetings, choppy egress recordings, and rising memory use; the reported delay measures how far behind live audio processing has become, with Python reporting seconds and Node.js milliseconds. Brief startup warnings may be harmless, particularly with the Silero plugin, but recurring warnings should be addressed by updating LiveKit Agents and turn detection, using the bundled inference VAD where appropriate, checking for blocked event loops caused by synchronous calls or CPU-heavy tasks, and ensuring adequate CPU and correctly configured self-hosted worker capacity. VAD metrics can help distinguish sustained CPU starvation from intermittent event-loop blocking, while increasing VAD threads, speech buffers, or idle processes is not recommended because these changes do not improve realtime processing and may worsen performance.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.