February 2024 Summaries

Serverless products inherently require usage-based billing models. However, building such systems is notoriously difficult due to the need for flexibility, scalability, and audibility. WarpStream, a serverless product, partnered with Orb to offload hard problems like event ingestion, processing, and storage from their engineering organization. To implement a simple yet effective pricing model, WarpStream created a time series event structure that separates "events" from "metrics," allowing for easy addition of new dimensions in the future. The system also employs idempotency keys to prevent double billing and incorporates checkpointing to optimize progress tracking. For its serverless product, WarpStream implemented an in-memory two-tiered aggregation architecture to measure network ingress and egress fees incurred by transmitting data through cloud environments, resulting in negligible overhead and a simplified architecture. By partnering with Orb, WarpStream was able to implement this usage-based billing model efficiently, reducing operational burdens and costs.

Middleware is a cloud observability company that uses AI to help customers identify, understand, and fix issues across their infrastructure. They rely on scalable streaming infrastructure to ingest and process tens of TiBs of telemetry every day. Initially, they used Apache Kafka as a scalable queue between event intake and downstream storage systems, but faced scalability issues with customer workloads being unpredictable and requiring frequent partition rebalancing. To mitigate this, they over-provisioned broker VMs, which drove up cloud infrastructure costs significantly. They also ran their critical infrastructure in three availability zones to ensure reliability, resulting in high inter-zone networking fees. Middleware's data ingestion and storage team then adopted Warpstream, a cost-effective alternative that saved them more than 85% compared to their previous setup, allowing them to focus on developing features for customers instead of managing scaling issues.