Company
Date Published
Author
Adnan Rahic
Word count
3501
Language
English
Hacker News points
None

Summary

Don't you really hate latency?` is the opening sentence of an article that discusses building applications on top of data warehouses, with a focus on achieving low latency when consuming data from BigQuery, Snowflake, Redshift, or other cloud-based data warehouses. The author aims to provide a crash course in data warehouse performance and highlight tools that help build responsive apps on top of these platforms. The article explains the basics of data warehouses, query latency, concurrency, and how to improve performance using techniques such as caching, materialized views, and pre-aggregations. It also introduces Cube, an open-source analytics API that can be used to create a semantic layer on top of any data warehouse, reducing latency and improving concurrency. The article concludes by sharing benchmarks that demonstrate the performance difference between querying BigQuery directly versus using Cube through a Node.js API.