October 2021 Summaries

This article discusses the complexity of maintaining data consistency in distributed environments and introduces conflict-free replicated data types (CRDTs) as a solution for resolving concurrent data changes. It covers strong and eventual consistency, replication conflicts, how Google Docs handles conflicts using operational transformations (OTs), and the advantages and disadvantages of CRDTs. The use cases for CRDTs include chat applications, productivity tools for online collaboration, ticketing systems, online auctions, presence awareness, and global data centers.

The partnership between Ably and Aerospike aims to solve intermittent connectivity, massive scale, and realtime processing challenges for organizations looking to deploy realtime digital experiences globally. As 5G rolls out and the amount of realtime data produced by users and companies continues to soar exponentially, everyone is racing to create and deploy the new generation of edge applications. Ably and Aerospike provide dependable, fully-managed, and instantly scalable platforms that make real-time, event-driven systems easier to build, maintain, and scale. This allows developers to focus on what makes their product special rather than handling outages on their hosting infrastructure. Both technologies have a strong focus on consistency and high reliability (99.999% uptime SLA) and are well suited for building distributed architectures with predictable performance.

An event indicates a state change and is used to signal interested parties that something has happened. Event-driven architecture (EDA) raises and responds to events, with an event producer triggering events and these reaching interested consumers via an event channel. The benefits of EDA include eliminating the need for a consumer to poll for updates, decoupling components for scalability, and providing realtime digital interaction capabilities. However, building a realtime event-driven architecture can be complex, costly, and time-consuming due to technical debt, performance concerns, data integrity issues, reliability challenges, and scalability limitations. To achieve best results, developers can use platforms like Ably that simplify the process of delivering feature-rich realtime digital experiences at scale with a unified platform for building powerful realtime applications.

The new Ably Kafka Connector provides a seamless integration between Apache Kafka and the Ably platform, enabling companies to distribute data from Kafka to internet-connected client devices in a fast, easy, dependable, and secure way. By 2025, it is projected that 30% of all data will be realtime, with much of this data being generated by consumers. The Ably Kafka Connector helps businesses leverage realtime data to provide personalized, instantaneous experiences for their customers. It extends Kafka without the pain and offers the same guarantees, messaging semantics, and characteristics as Kafka. This connector is now available in beta on the Confluent Hub as a verified Gold connector.

The future of events such as summits, conferences or concerts is brighter than ever thanks to hybrid events that combine in-person and online happenings. This allows event organizers to reach much larger, geographically distributed audiences. To provide a great ticket booking experience for these global audiences, it's crucial to build a dependable realtime ticket booking solution with characteristics such as scalability, realtime data flow, and data integrity. Key technologies include FastAPI, Apache Kafka on Confluent, ksqlDB, Ably, and Ngrok. The architecture of the ticket booking solution involves FastAPI for writing events to Kafka and consuming webhooks from Ably, Apache Kafka for internal event streaming and persistent storage, ksqlDB for stream processing, Ably for scalable and dependable data streaming to and from clients at the edge, and Ngrok as a proxy for ingesting webhooks sent from Ably into FastAPI.

This article series explores the implementation of a serverless and editable chat app using database-driven architectures. The first part covered the high-level architecture, while this second part delves into each section's implementation details to get the app working. Key points include setting up NuxtJS for the frontend, configuring PostgresDB, creating Lambda functions on the Ably integrations dashboard, and deploying the Ably Postgres connector. The architecture is designed to be scalable and serverless, with components deployed on various AWS services such as RDS, ECS, ECR, and Fargate. Further improvements are suggested for better performance and scalability.