Company
Date Published
Author
Igor Lukanin
Word count
2007
Language
English
Hacker News points
109

Summary

Databricks recently set a new data warehousing performance record in the 100 TB TPC-DS benchmark, claiming it was 2.7x faster and 12x better in terms of price performance compared to Snowflake. However, Snowflake responded with a blog post that critically reviewed Databricks' findings, reported different results for the same benchmark, and claimed comparable price/performance to Databricks. This back-and-forth between respected data vendors led to Databricks eliminating the anti-competitive DeWitt clause from their service terms, which Snowflake followed up with a similar update. The DeWitt clause is a provision in end-user license agreements that prevents publishing information about proprietary software without explicit approval from the vendor. This clause has become widespread in the industry and can have significant implications for benchmarking and competitive testing. As a result, some vendors have chosen to restrict or eliminate benchmarks with DeWitt clauses, while others permit it but require reciprocity. Open-source vendors do not typically include DeWitt clauses in their licenses, allowing users to publish benchmark results freely. Some cloud vendors, such as Amazon Web Services and CockroachDB Serverless, do not have DeWitt clauses and allow customers to perform and disclose benchmarks without restrictions. In contrast, some proprietary vendors like Cloudera Data Platform and Oracle restrict or require prior approval for publishing benchmark results. The absence of a DeWitt clause can facilitate open competition in the industry, while its presence can create barriers to entry. By exploring which vendors have DeWitt clauses and which do not, users can make informed decisions about whether to use their services for competitive testing and benchmarking purposes.