Stream & Go: News Feeds for Over 300 Million End Users

Stream is an API service that facilitates the creation of news feeds and activity streams, serving over 500 companies and reaching more than 300 million end users, including notable clients like Product Hunt and Under Armour. Initially developed in Amsterdam and later expanded to Boulder, Colorado, Stream has evolved significantly over three years, now boasting a team of over 15 developers. The infrastructure, originally built with Python and Cassandra, transitioned to Go and a custom key-value store based on RocksDB and Raft to enhance performance and simplify architecture, resulting in a significant reduction in latency and server requirements. Stream employs a hybrid approach of fanout-on-write and fanout-on-read to manage its data, offering optimal performance for both highly connected and sparse datasets. The company's stack incorporates a variety of technologies, including Go for its primary backend services, Python for machine learning and devops, and Django for its website, while also leveraging external services like AWS for hosting and Grafana for monitoring. Stream's real-time capabilities are powered by Go, Redis, and the Gorilla WebSocket library, with a focus on robust devops practices and comprehensive testing to ensure reliability and performance across multiple global regions.