Introducing Analytics Buckets

Supabase has introduced Analytics Buckets, a feature designed to manage and store large data sets in Supabase Storage, providing a solution for analytical workloads that differ from the capabilities of Postgres, which excels in handling transactional data. Analytics Buckets offer cost-effective storage, schema evolution, time travel, and full audit history, and they work in tandem with Postgres by storing historical data and running heavy analytical queries. Data is stored in Parquet files on S3, with Apache Iceberg handling metadata management, allowing for scalable and efficient querying using Iceberg-compatible tools like Apache Spark and Amazon Athena. The system supports automatic data replication and streaming via Supabase ETL, facilitating near real-time updates without manual intervention. This structure enables the separation and independent scaling of compute and storage, optimizing performance and cost by maintaining recent data in Postgres and archiving older data to Analytics Buckets, making it ideal for large-scale data analysis.