July 2025 Summaries

A recent analysis of AWS S3 Tables Iceberg compaction revealed a significant 20-30x cost disparity compared to other solutions, prompting AWS to reduce their prices by 10x. Despite this reduction, S3 Tables remains more expensive and demonstrates poor performance, particularly in compaction efficiency, which is crucial for managing the "small files problem" in large-scale data operations. The analysis highlights S3 Tables' limited control over compaction frequency and lack of observability tools, making it less suitable for real-time workloads. In contrast, Onehouse Table Optimizer offers a more cost-effective and efficient alternative, providing granular control over compaction settings and better performance, making it a compelling choice for users seeking a robust managed solution for Apache Iceberg tables.

AWS S3 Tables aims to simplify table maintenance for Apache Iceberg by offering a fully managed service with built-in support for storing tabular data at scale, leveraging a "table bucket" abstraction on Amazon S3. However, hands-on testing reveals significant drawbacks, such as long delays in compaction, limited observability, and costs up to 20-30 times higher than alternatives like AWS EMR. The default compaction settings result in performance degradation, especially for workloads requiring frequent updates. S3 Tables' limited control over compaction parameters further compounds these issues, making it less suitable for data-intensive workloads at scale. Despite being a significant milestone for the lakehouse ecosystem, S3 Tables may not be ready for high-performance needs, while Onehouse offers a more cost-effective and flexible alternative with advanced compaction techniques and comprehensive monitoring capabilities.