October 2021 Summaries

The text discusses serverless event stream processing with Aerospike, a distributed NoSQL database. It explains how to integrate Aerospike with AWS Lambda or Google Cloud Functions to build efficient cloud-native streaming pipelines. The process involves using the Aerospike Connect for Event Stream Processing (ESP) connector to convert Change Notification (CN) over XDR into HTTP/1 or HTTP/2 requests and stream them to downstream consumers. The blog provides a detailed example of how to build a streaming pipeline to stream Aerospike Change Notification events to AWS SQS using serverless functions. It also covers operationalizing the streaming pipeline, additional capabilities, potential use cases, and monitoring options.

The Aerospike Standup Vol. 40 is a newsletter for developers, featuring blog posts, community questions, documentation, releases, and events from the Aerospike team. This volume introduces Jupyter notebooks video tutorials and explainer videos on Flexible Reads in Strong Consistency mode. Community questions include queries about trial keys for Aerospike VS Scylla benchmark, hashing algorithm used in /opt/aerospike/smd/security.smd, and creating secondary indexes on multiple fields (composite key). New articles cover topics like Knowledge Base FAQs and XDR Retry Queue Throttling. Upcoming events include talks by Natalie at the Go Day conference and GopherCon. Release notes are also provided for October 22, 18, 15, and 13, along with a Docker image update.

Aerospike has published a Standup newsletter, which includes blog posts, community questions, documentation updates, and releases. The team is sharing updates every other week via email subscription for those interested in staying up-to-date on the latest developments. New tutorials and explainer videos have been added to help developers learn about Aerospike's features, including Flexible Reads in Strong Consistency mode. The Aerospike community has also shared trial keys for benchmarking and answered questions on various topics, such as hashing algorithms and secondary indexing. Recently published articles cover serverless event stream processing and the use of Connect for Event Stream Processing. Additionally, the team has released new knowledge base FAQs and upcoming events have been announced, including talks at conferences and a keynote speech at GopherCon. The release notes also list various updates and releases made to the Aerospike system.

Aerospike Inc. and Ably have partnered to integrate real-time data for IoT and mobile use cases, aiming to improve end customers' real-time digital experience. The combined solution leverages the Kafka ecosystem and large developer communities that both companies work with. It focuses on delivering comprehensive edge-to-core real-time event and data processing, particularly in handling high frequency workloads with increasing amounts of data created at the edge. The seamless integration between Ably's data capturing capabilities and Aerospike's data processing ensures minimal challenges for customers. This partnership is primarily aimed at IoT/connected devices and mobile use cases, with a focus on reducing friction, implementation time, and cost while maximizing any existing Kafka investment.

The rise of challenger banks like Revolut, Monzo and Starling Bank is disrupting the traditional banking industry as they use technology to gain market share. There are now over 250 challenger or neobanks globally, with the top 20 valued at more than $73 billion and having over 130 million customers. These fintechs are being propelled by a large influx of venture capital money. Challenger banks have become more attractive to customers due to their ease-of-use, no-fee accounts, early access to funds via banking apps, and the convenience of online banking during the pandemic. Traditional banks are trying to compete with new digital offerings, while non-bank companies like Amazon and Walmart also pose a threat by offering financial services to complement their core businesses. All these developments put pressure on legacy banks to innovate and expand their digital capabilities, as well as improve data access and transaction security.

The text discusses the use of Change Data Capture (CDC) in building event stream processing systems (ESP). It introduces Aerospike's Change Notification (CN), which is similar to CDC but differs in how changes are reported. The article also presents Aerospike Connect for Event Stream Processing (ESP), a tool that converts change notifications over XDR into HTTP/1 or HTTP/2 requests, enabling serverless event processing and integration with various cloud services like AWS Lambda and Google Cloud Functions. However, it clarifies that Connect for ESP is not a replacement for Kafka or Pulsar but can be used in certain use cases to avoid their challenges. The text also outlines several potential use cases of the tool, such as exporting Aerospike Change Notification messages to Elasticsearch and streaming data to AI/ML services like AWS SageMaker or Google AI. Finally, it announces the general availability of Aerospike Connect for ESP for Change Notification and encourages users to download and try it out.

Aerospike Connect for Event Stream Processing (ESP) is a solution that converts Change Data Capture notifications into HTTP/1 or HTTP/2 requests, enabling integration with downstream systems such as AWS Lambda, Google Cloud Functions, ElasticSearch, and Splunk. It offers flexible deployment options in the cloud or on-premises and can be used to trigger serverless event processing, build cloud-native data pipelines, and enable real-time ad campaigns. Connect for ESP is designed to work with Aerospike's Cross-Datacenter Replication (XDR) system, which publishes changes as they occur, allowing for efficient data exchange and at-least-once delivery guarantee. While it can be used in place of Kafka or Pulsar for certain use cases, it also has limitations such as no message retention, topics, or partitions, and no schema registry support.

The Aerospike Standup newsletter, aimed at developers, covers blog posts, community questions, documentation updates, releases, and events from the Aerospike team. Recently, they released Server v5.7 with new secondary index features for improved memory efficiency, faster queries, and a better garbage collection system. They have also added more XDR-related content to their knowledge base. Upcoming events include talks at Go Day conference and GopherCon.

We've recently released Server v5.7, featuring improved memory efficiency, faster queries, and a new garbage collection system; the release includes changes to configuration parameter defaults, so it's essential to review the Release Notes for updates. The Aerospike team has expanded its knowledge base with XDR-related content, providing more educational resources for developers. Additionally, we'll be sharing updates every other week through our newsletter, which is available for subscription, and upcoming events include a talk at Go Day conference on November 16 and a keynote at GopherCon on December 7.

In a recent webinar, experts from FreeWheel, AWS, and Aerospike discussed how real-time data platforms can enable rapid growth in the Ad Tech industry. Beeswax, a subsidiary of Freewheel, chose to build their architecture on AWS due to its ability to scale up and down quickly, which is crucial for handling high volumes of ad requests per second. Aerospike's real-time database was also critical for achieving low latency in bidding systems. Managing costs at scale was another key factor, with Beeswax leveraging AWS to match compute capacity with demand and reduce server footprint by 80%. The webinar covered four ways a real-time data platform can enable 10X growth in Ad Tech, including achieving speed at scale and managing costs at scale.

Beeswax, a subsidiary of Freewheel, a Comcast company, recognized the need to choose the correct technology from the start due to Ad Tech's speed and scaling requirements. To achieve high performance at scale, Beeswax built its architecture in the cloud from the get-go, using a real-time database like Aerospike's to handle the sheer volume of incoming requests and provide low latency for bidding systems. This allowed them to spin up compute quickly and access compute capacity as needed, while also leveraging the scalability of AWS to manage costs at scale by scaling up and down based on load coming into the system. By using a cloud solution like AWS and Aerospike, Beeswax was able to reduce complexity and manpower needed to manage the environment, resulting in 80% reduction in server footprint.