How Modal uses ClickHouse to power real-time observability for AI workloads

Modal utilizes ClickHouse Cloud to enhance real-time observability for AI workloads across numerous GPUs and containers, overcoming previous scaling issues with data reads and writes. ClickHouse Cloud enables Modal to ingest 1-2 million events per minute and manage around 500 billion logs while maintaining sub-second query speeds, which is crucial for AI infrastructure that supports large-scale GPU workloads for training, inference, and batch processing. Through a seamless Python SDK, developers can deploy workloads without dealing with the complexity behind the scenes. Modal's use of ClickHouse has resulted in the development of several real-time dashboards that offer users detailed insights into function performance, such as execution time and latency trends. These dashboards, powered by a single ClickHouse table, facilitate efficient data querying and provide full lifecycle visibility of function calls. As Modal continues to grow, with event ingestion doubling to 2 million per minute, the team is exploring new features like a billing API and a visual function call graph to further enhance user experience and performance.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.