Company
Date Published
Author
Daniel Jagielski, Gil Friedlis, Matt Mangia
Word count
2381
Language
English
Hacker News points
None

Summary

The text discusses the development of a security platform called Risk Engine at Tesco PLC's Identity API. The initial version of Risk Engine was built using JMS and had limitations such as not being able to handle high traffic volumes, causing performance issues and data loss. To address these challenges, the team rebuilt the module from scratch using Apache Kafka and event streaming, with a new design that utilized finer granularity of services, minimum value aggregates, and data locality. This approach resulted in improved performance, scalability, and availability of the platform, eliminating single points of failure and reducing memory and CPU usage. The journey to develop this platform is ongoing, but it has already provided valuable lessons on the importance of proper data analysis, design principles, and the use of event streaming technologies like Kafka Streams and ksqlDB.