A new way to create intermediate Data Sources in Tinybird

Tinybird has launched Copy Pipes as a general availability feature, allowing users to seamlessly transfer the results of Tinybird Pipes into new or existing Data Sources, enhancing data management and application building capabilities without the need for APIs. Initially released in beta, Copy Pipes have quickly become popular, accounting for over 10% of the data written in Tinybird, and they support various use cases such as consolidated inventory snapshots, real-time OLTP enhancements, and change data capture (CDC). Unlike Materialized Views, which can be inappropriate for certain use cases, Copy Pipes provide a more efficient method for creating periodic data snapshots or deduplicating records. Users can now create, trigger, pause, and resume Copy Pipe jobs via Tinybird's UI and CLI, offering flexibility in scheduling and parameter usage. The official launch aims to improve data processing efficiency while addressing the needs of users seeking to streamline intermediate data handling in their data engineering workflows.