Company
Date Published
Author
Kevin White
Word count
450
Language
English
Hacker News points
None

Summary

Manage, a technology company specializing in programmatic mobile marketing and advertising, generates over a terabyte of data daily and processes more than 30 billion bid requests. To manage this scale, they initially used MySQL but later switched to Hadoop with Apache Hive and Kafka, only to find that Hive was slow. In search of a faster solution, Manage turned to SingleStore, which enabled them to reduce the delay in data freshness from two hours to 10-15 minutes, allowing for real-time analytics and ad-hoc queries on log-level data within seconds. With SingleStore Streamliner, an Apache Spark solution, they stream log data from Kafka, store it in a columnstore, and aggregate it into summary tables, providing a highly scalable and real-time data pipeline.