BigQuery connector for ClickPipes is now in Private Preview

The BigQuery connector for ClickPipes, now in Private Preview, streamlines the process of loading data from BigQuery into ClickHouse Cloud, allowing for fast exploration and prototyping by automating data migration tasks such as schema mapping and type conversion. This integration addresses challenges like the manual effort previously required for exporting data and configuring pipelines, offering efficient bulk data loads and fine-grained control over datasets and tables. Built on the PeerDB platform, it supports continuous ingestion and is designed to enhance the performance of interactive queries by functioning as a speed-layer atop BigQuery, with potential future capabilities for incremental data syncs. The connector utilizes BigQuery's native export jobs to manage data transfers cost-effectively and requires specific setup in Google Cloud, including a dedicated staging bucket. As the service evolves, its development is guided by customer feedback, with opportunities for design partnerships to refine features like incremental syncs, which remain a priority due to customer demand.