Blog

The post discusses the engineering advancements made by Tinybird to enhance its branching capabilities, allowing developers to test changes using real production data without copying it, and to deploy updates with zero downtime. Each branch operates as a full Tinybird project within an ephemeral cloud environment, featuring its own ClickHouse database and API surface, which isolates it from production. Tinybird utilizes a zero-copy partition sharing method that leverages hardlinks rather than data duplication to avoid additional storage costs. The engineering solution ensures that branches can safely access and manipulate data by using immutable ClickHouse parts, thus preventing any corruption to the production environment. Connectors, such as Kafka and S3, are isolated by creating new consumer groups and sampling data without affecting production sync cursors. The process of deploying changes to production involves a series of API calls managed by Tinybird's CLI, streamlining schema migrations and ensuring continuity during transitions. This architecture supports seamless integration into CI/CD workflows and allows developers to iterate safely and efficiently.