November 2024 Summaries

The Ably Pub/Sub architecture ensures reliable and efficient message delivery across a global network of servers, with a focus on data integrity. Each region in Ably can operate independently but also coordinates with other regions to share and replicate messages globally. Messages are processed and stored by primary locations before being acknowledged, while secondary locations store backup copies. In case of regional failures, the system ensures seamless fallback to another region without affecting global availability. The architecture supports exactly-once delivery through idempotent publishing and message delivery on SDKs, ensuring no duplicates or gaps in the message stream. Additionally, messages maintain their order relative to other messages on the same WebSocket connection, providing causal consistency for each client. Overall, Ably's Pub/Sub system is designed to "just work" with a focus on data integrity and performance.

Scalability is a crucial aspect for any system that needs to handle large or unpredictable workloads, and it's an essential element of modern cloud platforms. It refers to the ability of a system to change in size or scale to accommodate varying workloads. Ably has been designed around these principles, enabling it to handle vast realtime messaging workloads across multiple dimensions. The platform achieves scalability through horizontal scaling strategies, which involve having more components instead of larger ones. This approach supports the arbitrary and elastic scalability needed by platforms like Ably.

Maintaining message order in large-scale chat applications is a complex distributed computing problem that becomes more challenging as the number of users, traffic, and backend servers increases. Coordinating multiple chat users, managing retries and timeouts, and synchronizing across devices and users are some of the main issues that arise when scaling chat applications. To address these challenges, developers can use techniques such as unique identifiers and sequencing, buffering, windowing, and watermarking, concurrency and load distribution, handling retries and deduplication, and other architectural considerations. Ultimately, teams must balance the need for control and customization against resource limitations when deciding whether to build chat functionality in-house or use existing platforms and libraries.

Consumption-based pricing has become popular among SaaS and PaaS businesses, allowing customers to pay only for the resources they use. This approach offers flexibility and reduces waste, with pioneers like Slack and AWS successfully adopting it. However, not all consumption-based models are created equal. The Monthly Active Users (MAU) model, while appealing in its simplicity, often leads to inefficiencies and unexpected costs. This article explores the strengths and weaknesses of various consumption-based pricing strategies, and shares the journey Ably has taken to identify a model that truly prioritizes customer value.

Apache Kafka is a popular real-time data streaming platform known for its ability to handle massive volumes of data with minimal latency. However, scaling Kafka to manage thousands of client connections can be complex and costly due to its architecture being designed for efficient data processing rather than handling high volumes of connections. Many organizations offload client connections to a connection management service to optimize scaling for Kafka. Combining Kafka with WebSockets allows developers to scale Kafka efficiently for throughput by using a WebSocket layer to handle client connections, offering real-time bi-directional communication and independent scaling for data ingestion and delivery. This approach can lead to significant cost savings as the infrastructure for Kafka is scaled only for the data it processes, not for the volume of connections.