How Respan is scaling LLM observability with ClickHouse Cloud

Respan, formerly Keywords AI, leverages ClickHouse Cloud to handle high-throughput observability for large language model (LLM) applications, processing around 50 million events per day. Initially using a Postgres-based system, Respan faced challenges as their workload increased to 50-100 requests per second, prompting a migration to ClickHouse Cloud to better manage real-time data ingestion and analytics. The platform's backend, now optimized for high-speed analytics, employs incremental materialized views and compact data storage to maintain dashboard performance while minimizing query costs. This architecture supports both real-time metrics and user metadata, allowing efficient data aggregation and refreshing without the need for constant recomputation. Raymond Huang, co-founder and CTO, highlights the role of ClickHouse in simplifying operations through its indexing model, which ensures scalability and efficient querying without frequent architectural changes. Respan’s observability stack, now underpinned by ClickHouse, provides robust support for tracing LLM workflows, enabling detailed tracking of individual requests within the system.