July 2024 Summaries

Ably provides realtime messaging services with low latency. A new customer experienced unusually high variance in message latency between Europe and South East Asia. Initial investigation revealed that the issue was not due to processing delays but propagation delays on the TCP/IP connection. The problem was traced back to the tcp_slow_start_after_idle setting in the Linux TCP stack, which resets the congestion window after an idle period. Disabling this setting within Docker containers eliminated the latency tails and improved message delivery times for the customer's bursty traffic. This highlights the importance of understanding and optimizing network stack configurations tailored to specific use cases.

Apache Kafka is a popular choice for real-time data streaming due to its scalability, fault tolerance, and low latency capabilities. Kafka Connectors act as bridges between event-driven and non-event-driven technologies, enabling the streaming of data between systems. However, traditional approaches to scaling Kafka Connectors often prioritize throughput over latency optimization, which can negatively impact applications that rely on low latency for success. To address this issue, Ably developed its own Kafka Connector and conducted research to determine the optimal configuration for achieving minimal latency under moderate load scenarios. The company used distributed tracing with OpenTelemetry to measure end-to-end latency and identify bottlenecks in their system. By implementing a comprehensive approach using distributed tracing, they were able to pinpoint latency bottlenecks and understand the impact of various optimizations on overall performance. The research revealed that increasing the number of partitions did not significantly improve latency, while adjusting the tasks.max setting could help balance concurrency and resource utilization. Using simpler converters like the built-in String converter reduced overhead associated with serialization and deserialization. Experimenting with batching intervals showed that even small batching intervals increased latency, so avoiding batching altogether was the most effective approach to maintain low latency in their specific use case. Increasing the number of threads in the connector's internal thread pool significantly reduced latency by allowing more messages to be processed in parallel but came with trade-offs such as potential issues with message ordering. The insights and optimizations explored can be applied broadly to any Kafka Connector to improve performance and reduce latency, emphasizing the importance of understanding and optimizing built-in Kafka configuration settings and connector-specific settings.

Ably has launched a private beta of its new chat product, Ably Chat, designed for various realtime applications such as global livestreams and customer support chats. The platform is built on Ably's reliable global platform with proven performance guarantees and scalability. Ably Chat comes with purpose-built APIs for chat features like 1:1, 1:many, many:1, and many:many chat rooms, user messaging, online status tracking, typing indicators, room level reactions, and more. The platform is flexible, dependable by design, and cost-effective, allowing developers to create efficient chat architectures for their applications.

Ably is transitioning from its legacy status page to an upgraded one, hosted on the industry-standard platform Atlassian Statuspage. The new status page offers improved functionality, enhanced features, and better integration with popular tools and platforms. Key benefits include configurable multi-channel notifications, seamless integrations, and Ably's availability as a third party component. This change aims to provide users with better communication, increased transparency, and enhanced workflow integration. The current status page will be available until 23rd July 2024, after which it will point to the new one.

Fan expectations have evolved due to social platforms, leading fans to seek ongoing interactive experiences that connect them with creators and other fans. This presents a significant opportunity for fan-centric businesses to strengthen relationships and generate new revenue streams. However, technological challenges can make these experiences economically infeasible, particularly as costs grow exponentially with audience size. To address this issue, companies must design their systems to scale linearly rather than exponentially, using strategies such as batching, aggregating, and partitioning messages. By implementing these architectural patterns, fan-centric organizations can deliver economically viable realtime fan experiences at scale.