Why we ditched Prometheus for autoscaling (and don't miss it)

Tinybird developed a custom autoscaling system leveraging Kubernetes Event-Driven Autoscaling (KEDA) to address the unpredictability of real-time analytics workloads, which can experience up to tenfold traffic spikes during events like product launches. Traditional autoscaling methods, based on CPU and memory metrics, were inadequate due to their reactive nature and latency in scaling decisions. By integrating KEDA directly with Tinybird’s real-time metrics API, the company eliminated the delays associated with Prometheus-based metrics scraping and instead scaled based on live ingestion data. This approach allowed for more responsive and reliable scaling by focusing on critical metrics like Kafka lag, which better reflected actual data processing demands. The system adapted to varying traffic patterns and ensured stability by tuning stabilization windows and combining multiple triggers, such as custom metrics and CPU utilization. This overhaul not only optimized infrastructure performance and cost-effectiveness but also provided a feedback loop that enhanced the overall product's reliability and usability.