September 2020 Summaries

AppMonet, a New York-based company, connects mobile app developers with advertisers through video ads, utilizing a unified SDK to support major video brands and maximize advertising demand sources. By employing AI for content optimization and offering diverse ad formats, AppMonet has gained popularity among global applications. The company transitioned from Erlang mnesia to ScyllaDB Cloud to scale its infrastructure, choosing ScyllaDB for its flexibility and performance, particularly for their read-heavy workload. Hosted on AWS, ScyllaDB Cloud supports AppMonet’s real-time writes and batch updates, with workload prioritization to prevent analytics from affecting production. AppMonet benefits from ScyllaDB’s low-latency data retrieval, critical for efficient ad bidding, and its scalability, allowing AppMonet to handle significant query loads with minimal performance drop. As AppMonet continues to grow, ScyllaDB's multi-datacenter replication feature aligns with their plans for global expansion.

A blog post from ScyllaDB University introduces a lesson focused on connecting a Scala application to a ScyllaDB cluster using the Phantom library, which is a Scala-idiomatic wrapper over the standard Java driver. The lesson demonstrates coding a sample application that interacts with a database, specifically the Mutant Catalog table in ScyllaDB, showcasing operations such as displaying, inserting, and deleting data while emphasizing the use of strongly-typed domain objects for maintainability and compile-time query verification. The process involves setting up a ScyllaDB cluster, creating a keyspace and table, and running the application within a Docker container. This lesson is part of a series aimed at educating developers on NoSQL database development and administration with Scala, with future lessons promising to delve deeper into application structure using the Phantom library.

ScyllaDB has traditionally relied on seed nodes to manage cluster discovery and gossip information propagation, but this architecture has introduced complexities and operational challenges due to the need for special roles and configurations. Seed nodes do not participate in the bootstrap process, leading to user confusion and necessitating careful documentation to avoid errors. To address these issues, ScyllaDB aims to eliminate seed nodes in favor of a more symmetric node architecture, thereby simplifying configuration and improving robustness. The new approach involves changes to gossip shadow and normal round communication to make the process more deterministic, along with the introduction of a new synchronous RPC message to streamline communication during the gossip shadow round. These changes ensure all nodes, both seed and non-seed, must bootstrap when joining the cluster, reducing configuration errors and making the system easier to manage. These improvements are part of ScyllaDB's upcoming 4.3 release, marking a shift to a fully seedless architecture.

AWS Outposts offers a fully managed on-premises extension of Amazon Web Services, allowing users to run various AWS services locally, such as EC2 instances, EBS, and container-based services like Amazon EKS. While not all AWS services, including Amazon DynamoDB, are available on AWS Outposts, ScyllaDB provides an alternative with its DynamoDB-compatible API, Alternator, enabling users to run DynamoDB workloads on AWS Outposts. ScyllaDB Cloud, a fully managed database-as-a-service, supports both the DynamoDB API and Cassandra Query Language (CQL) for on-premises deployments, and is certified as "AWS Outposts ready." Users can leverage ScyllaDB Cloud in a hybrid setup across on-prem and public AWS regions, and can also utilize ScyllaDB Enterprise or Open Source versions for self-managed database solutions. ScyllaDB offers resources and support through its AWS Partner Network, ScyllaDB University, and community channels to facilitate the integration and management of ScyllaDB on AWS Outposts.

The blog post outlines a hands-on project called CarePet, which serves as an example for developers to create an Internet of Things (IoT) application using ScyllaDB as the backend datastore. The project involves developing an application in Go that tracks pets' health indicators through sensors attached to a pet's collar and comprises three main components: a sensor data collection system, a web application for data analysis, and a database migration tool. The application aims to handle data for up to 10 million pets, with a significant focus on high write throughput and low read latency to ensure scalability and performance. It demonstrates the full development process, including requirement gathering, data modeling, and application deployment, while emphasizing the use of Docker for running a ScyllaDB cluster and the Go programming language for building the application's components. The post also touches on the code structure, implementation details, and future plans for expanding the use case, encouraging developers to explore the associated GitHub repository and additional learning resources on ScyllaDB.

The blog post discusses the process of migrating data from Amazon DynamoDB to ScyllaDB using the ScyllaDB Migrator tool, which now includes live replication capabilities. This new feature leverages DynamoDB Streams, introduced in 2014, to capture and replicate table modifications in real-time using Spark Streaming. The process involves an initial snapshot transfer of the source table, after which the migrator consumes the DynamoDB Stream to apply continuous changes to the ScyllaDB table, ensuring no data loss and eventual synchronization between the source and target tables. The post provides a detailed walkthrough of configuring the migration process, highlighting the necessary steps and resources needed to complete it within DynamoDB Streams' 24-hour retention limit. Additionally, it demonstrates a test run using a load generation tool to validate the functionality, confirming that the contents of both the source and target tables remain identical after migration.