Company
Date Published
Author
Joseph Lopez
Word count
1965
Language
English
Hacker News points
None

Summary

Harness has successfully utilized Looker's aggregate tables to optimize dashboard load times and reduce data processing costs for large customers, achieving efficiency gains of up to 1000 times. Aggregate tables, a type of Persistent Derived Table (PDT), allow the creation of smaller rollup tables that enhance data engineering flexibility and scalability by intelligently selecting the smallest table to satisfy queries. This approach significantly reduces the amount of data processed, resulting in faster dashboards and cost savings. Specifically, Harness applied this strategy to its Cloud Cost Management module, improving dashboard performance from minutes to seconds and reducing data processing to a fraction of its original size, thus providing a more efficient and cost-effective solution for its customers.