Company
Date Published
Author
Lucia Cerchie, Jay Kreps, Josep Prat
Word count
1811
Language
English
Hacker News points
84

Summary

Kafka can be used as a primary store for data, but it's not necessarily crazy to do so, especially since it was designed with storage in mind. However, treating Kafka as a primary store does require running the system correctly and knowing its limitations. Unlike traditional messaging systems, Kafka is more akin to a distributed file system or database, with features like replicated logs, high-throughput reads, and concurrent writes. While Kafka can be used for stream processing, it's not intended to replace databases entirely, but rather serves as a commit log or storage layer for derived indexes or views. Its true mission is to make streams of data and stream processing a mainstream development paradigm, acting as a central nervous system for modern digital businesses.