November 2019 Summaries

SaaS companies are increasingly leveraging realtime technology to meet evolving user expectations for live, on-demand interactions and seamless integrations between various applications, as exemplified by platforms like HubSpot, Slack, and Zoom. The adoption of realtime frameworks and protocols, such as WebSockets, allows these companies to provide immediate, sub-100ms interactions in their apps, enhancing experiences like live updates and chat functionalities. The complexity of building and maintaining such infrastructure can be mitigated by using specialized providers like Ably, which offer cloud infrastructure and APIs that simplify realtime engineering and allow developers to focus on core product features. This approach, known as the Realtime Snowball Effect, involves adopting, optimizing, and accelerating the use of realtime services, ultimately transforming how products are developed and enhancing customer experiences. As user expectations continue to shift towards realtime interactions, SaaS companies that effectively integrate these technologies will be better positioned to thrive in the future.

Ably has developed a distributed real-time messaging platform and outlines the essential skills and knowledge for a distributed systems engineer. Key areas of expertise include distinguishing between microservices and truly distributed systems, understanding hash rings, and grasping the significance of gossip protocols and consensus algorithms, which are crucial for maintaining real-time state changes in distributed environments. Engineers must also navigate the trade-offs between consistency and availability, often opting for eventual consistency through conflict-free replicated data types, and must possess a deep understanding of networking protocols across all layers, including HTTP, WebSockets, and TCP. Ably emphasizes that a solid foundation in systems engineering, including programming, design patterns, and infrastructure management, is vital for tackling the challenges inherent in global distributed systems. The company offers cloud infrastructure and APIs to aid developers in creating scalable real-time applications and is actively expanding its team with multiple open positions in distributed systems engineering.

Ably introduces Channel Rewind, an optimization for its Pub/Sub Channels that simplifies the process of retrieving realtime messages sent over a channel prior to connection. With this feature, developers can now connect to a channel, instantly load the state, and subscribe to all future realtime updates with a single API interaction. Channel Rewind is useful in various scenarios such as location tracking, weather updates, chat applications, IoT devices, and more. It also works well with Message Delta Compression and State Persistence features offered by Ably.

The article discusses the challenges developers face when accessing and using public APIs for real-time open data, focusing on live public transport updates. It highlights that while examples like Transport for London showcase the benefits of real-time open data, such large-scale success stories are rare due to various obstacles. These include polling taking up time and resources, fragmented data structures, and some data sets being more open than others. The article suggests solutions such as using push-based systems instead of pull-based ones, standardizing data structures across sectors, and imposing generous limits on public data providers.