Sinks: Export your data to S3, GCS, and Kafka

Tinybird Forward has expanded its data export capabilities, introducing sinks for Amazon S3, Google Cloud Storage, and Kafka, enabling users to export processed data from Tinybird pipes to these external platforms either on a schedule or on-demand. Sinks are extensions of Tinybird pipes, allowing the results of SQL queries to be written to files or streams rather than exposed as API endpoints. These sinks support various file formats, including CSV, NDJSON, and Parquet, with options for advanced file partitioning and compression tailored to the needs of downstream systems like data warehouses and analytics tools. The new functionality includes enhanced partitioning strategies that facilitate efficient data organization, such as creating Hive-style partitioned structures for compatibility with tools like Athena, BigQuery, or Spark. Execution of sinks can be managed through cron expressions for scheduled runs or triggered manually for on-demand operations, with comprehensive logging and monitoring features to track performance and data transfer costs. These sinks are available to users on Developer and Enterprise plans, with detailed setup guides provided for each destination type.