Home / Companies / Tinybird / Blog / Post Details
Content Deep Dive

Compute-compute separation for faster populates

Blog post from Tinybird

Post Details
Company
Date Published
Author
Jesus Botella
Word Count
620
Language
English
Hacker News Points
-
Summary

Tinybird's recent advancements in Compute-Compute Separation for Populates enhance performance by allowing additional compute resources to be provisioned on-demand, specifically for demanding jobs like big backfills or heavy transformations, without affecting primary workloads or incurring costs for idle capacity. The improvements include faster start times with replicas ready four times quicker, optimized settings for maximum throughput, increased control via the Tinybird CLI, and better observability of job progress. The feature now supports Google Cloud Platform (GCP) alongside AWS, aligning with Tinybird's mission to enable faster deployments and efficient resource usage. The technical infrastructure involved multi-cloud support using a strategy pattern, Infrastructure as Code with Pulumi for better integration with Python services, and asynchronous setup with Kubernetes jobs to maintain API responsiveness. Performance optimizations focused on reducing startup times and tuning workload-specific settings, resulting in a scalable system capable of handling intensive operations without impacting reliability or performance.