June 2021 Summaries

DataStax Astra DB is a multi-cloud database-as-a-service built on Apache Cassandra™ that complements cloud strategies with its cloud native nature, autoscaling capabilities, and reduced total cost of ownership (TCO). Security is crucial for enterprises adopting cloud services, especially when it comes to managing their most valuable asset: data. Astra DB offers an IP Access List feature that enables users to custom configure IPs and CIDR blocks that can access hosted resources, allowing enterprises to restrict network access from select office and home networks. This feature, combined with other security measures like Role-Based Access Control (RBAC), helps establish a robust security practice. Astra DB is the fastest and easiest way to set up secure Cassandra clusters, offering an intuitive UI workflow and API configuration options.

DataStax is developing a messaging solution to complement its existing database ecosystem, particularly for microservices-based architectures that use a message bus. The company has evaluated Apache Kafka and found it lacking in four areas, leading them to consider Apache Pulsar as an alternative. This new service aims to provide a comprehensive solution for developers and architects working with DataStax Astra Cassandra-as-a-service.

The transition from Kafka to Apache Pulsar can be complex and risky, often resulting in organizations being locked into suboptimal situations. However, mitigating these risks is possible by making the transition process iterative and breaking it down into manageable steps. The ecosystem supporting this transition includes built-in connectors for pulling/pushing data between Pulsar and Kafka topics, as well as the Kafka Connect Adaptor (KCA) Sink and Source. Apache Pulsar 2.8 offers improvements to the Java Pulsar Schema API and the Pulsar IO API that simplify schema handling in Sinks. These enhancements make it easier for users to develop Pulsar IO Sinks without being tied to a particular schema at build time, and support KeyValue messages seamlessly. The Kafka Connect Adaptor completes the compatibility ecosystem between Kafka and Pulsar, allowing an iterative transition from one system to another while supporting native Kafka clients with Pulsar and data transfer between two systems.

DataStax Astra introduces Astra Streaming, a massively scalable and highly performant event streaming and cloud messaging platform built on top of Apache Pulsar. This public beta offering addresses the data in motion requirements for modern data apps. Astra Streaming provides developers with a complete toolkit for data streaming, asynchronous communication between services, and data integration with Astra DB. It supports Fast JMS for Pulsar, allowing enterprises to modernize their JMS applications and connect them to Apache Pulsar's full capabilities without undertaking long migration projects. The platform is fully managed on AWS, GCP, or Azure, ensuring low latency and ease of deployment.

NoSQL databases are designed to support cloud applications and overcome limitations of relational databases (RDBMSs). They come in various forms, including Apache Cassandra®, MongoDB, Apache HBase™, and Couchbase. These databases differ from RDBMSs in their architecture, data model, distribution model, replication design, and development model. NoSQL databases are suitable for large volumes of structured, semi-structured, or unstructured data, high scalability, continuous availability, big data processing, and real-time analytics. Key decision points on when to use RDBMSs or NoSQL include the type of application (centralized vs decentralized), data types, velocity of data, data distribution, and storage capacity. Cassandra is a leading NoSQL database that offers benefits such as high availability, distributed architecture, wide-column store data model, efficient data replication, and user-friendly query language. Benchmarks show that Cassandra outperforms other popular NoSQL databases like MongoDB, HBase, and Couchbase in terms of speed and performance for mixed operational and analytic workloads.

The text compares the architecture and implementation of Cassandra with other top distributed NoSQL databases such as Couchbase, HBase, and MongoDB. It highlights that Cassandra incorporates several best practices for performance, making it a popular choice among leading companies worldwide. Unlike its competitors, every Cassandra machine handles an equal share of system activities, resulting in simpler installation and operation. Additionally, Cassandra's masterless design delivers lower latency and uninterrupted uptime. The text also discusses the benefits of Cassandra's log-structured engine, which avoids overwrites to turn updates into sequential I/O operations on both hard disks (HDD) and solid-state disks (SSD). Furthermore, it mentions that Cassandra supports indexes and increasingly complex queries, unlike some other NoSQL databases. The text concludes by stating that when comparing each NoSQL database option, Cassandra stands out as the clear winner in terms of distribution, storage, queries, scaling, updates, and replication.

Cassandra stands out among top distributed NoSQL competitors due to its architecture and implementation that incorporate several best practices affecting performance. Unlike other systems, every Cassandra machine handles a proportionate share of system activities, making it simpler to install and operate. Its masterless design delivers lower latency and uninterrupted uptime. Cassandra's log-structured engine avoids overwrites on both hard disks (HDD) and solid-state disks (SSD), preventing performance degradation as the dataset size exceeds RAM. In comparison to MongoDB, Couchbase, and HBase, Cassandra supports more complex queries, offers better developer productivity with its Cassandra Query Language (CQL), and has a robust implementation for handling large data sets and replication scenarios. Overall, Cassandra's architecture and implementation make it the clear winner in distribution, storage, queries, scaling, updates, and replication among top NoSQL databases.

Apache Cassandra is an open-source, distributed NoSQL database used by top companies worldwide for various applications. Developed at Facebook and released in 2008, it became a top-level Apache Foundation project in 2010. Companies like Apple, Instagram, Uber, Spotify, Twitter, Cisco, Rackspace, eBay, and Netflix use Cassandra due to its numerous benefits, including handling structured, semi-structured, and unstructured data, automatic workload and data balancing, operational simplicity, low overhead, and support for hybrid and multi-cloud environments. Cassandra is commonly used in e-commerce inventory management, personalization and recommendation engines, digital banking, IoT and edge computing needs, fraud detection, identity authentication, and other applications. Many prominent companies have adopted Cassandra to improve their services and meet customer expectations. Some of the most common use cases include Netflix, Soundcloud, and Instagram, which leverage Cassandra's powerful features for audit logging, dashboard customization, data replication, and global scaling. Cassandra is a suitable choice for companies requiring large-volume data management, always-on availability, high fault tolerance, easy scalability, seamless replication, and performance optimization in cloud-native applications or hybrid/multi-cloud environments.

Apache Cassandra is an open-source, distributed NoSQL database widely used by top companies for various applications due to its numerous advantages over rival systems. Developed at Facebook and released in 2008, it became a part of the Apache Foundation project in 2010. Companies like Apple, Instagram, Uber, Spotify, Twitter, Cisco, Rackspace, eBay, and Netflix use Cassandra for its flexibility to handle structured, semi-structured, and unstructured data, automatic workload and data balancing, operational simplicity, low overhead, and support for hybrid and multi-cloud environments. Cassandra is commonly used in e-commerce and inventory management, personalization and recommendation engines, digital banking, internet of things (IoT) and edge computing needs, fraud detection, identity authentication, and other applications. It has helped prominent brands like Netflix, Uber, Instagram, Reddit, Soundcloud, and more to expand their services. Cassandra's built-in audit logging capability is used by companies for tracking significant changes in the database, ensuring regulatory compliance, security standards, and uncovering the root cause of bugs. It also provides a solid foundation for dashboards, as seen with Soundcloud's customer dashboard. Additionally, Cassandra's peer-to-peer, distributed architecture enables data replication across geographically dispersed data centers, increasing fault tolerance, reliability, and availability without compromising performance. Overall, Apache Cassandra is a powerful and flexible NoSQL database that can help any company manage large volumes of data while ensuring high fault tolerance, easy scalability, seamless replication, and always-on availability.

The text discusses the increasing use of real-time data analysis and decision making in various industries, including transportation, healthcare, finance, and manufacturing. It highlights the importance of technologies like streaming, messaging, live feeds, and event-driven data. DataStax Luna Streaming, a subscription service based on open-source Apache Pulsar, is mentioned as an emerging solution for real-time data processing. A performance test comparing Luna Streaming (Pulsar) and Kafka clusters with Kubernetes revealed that Luna Streaming had higher average throughput in all the OpenMessaging Benchmark testing workloads performed. The text concludes by offering a free trial of Astra Streaming, DataStax's fully managed Apache Pulsar service, and inviting readers to attend an upcoming webinar with Overstock.com on June 29 at 9 am PT to learn more about using Apache Pulsar in ecommerce.

Real-time data processing is becoming increasingly prevalent across various industries, enabling real-time decision making and analysis. Data streaming technologies such as Apache Kafka and Apache Pulsar are being used to capture, analyze, and react to live data feeds. In a recent performance test, Luna Streaming (based on Apache Pulsar) outperformed Kafka clusters in terms of average throughput across various workloads. Assuming linear growth of an enterprise's streaming data needs over three years, using Luna Streaming could result in 33% savings in infrastructure costs compared to Kafka. For projects with significant complexity but limited raw throughput requirements, the infrastructure savings can be as high as 75%.

Traversals Analytics and Intelligence GmbH, a German company specializing in data analytics for cybersecurity, fraud detection, competitor intelligence, and real-time alerting, has chosen DataStax Enterprise Graph as its foundation. The company was already familiar with TitanDB, which DSE Graph is based on, and found it to be the best solution that met all of Traversals' requirements—it could process data in real time and at scale, and it could manage data in a distributed way. Today, Traversals delivers services around security intelligence, competitor analysis, and real-time disaster alerting using DataStax Enterprise Graph and the DataStax Kubernetes Operator for real-time analytics and alerting with the speed and throughput that its customers expect.

DataStax Astra has introduced serverless databases with automatic hourly backups, stored for 20 days. Users can restore their database to a previous state within 20 calendar days of the desired backup by contacting DataStax support. Backup and restore capabilities are set to expand in the future, allowing users to manage data snapshots directly from the Astra console. For more information or assistance, contact the Astra team at [email protected].

Apache Pulsar is a distributed messaging system used by hundreds of companies to solve various problems at scale. Nutanix, a leader in cloud software and hyperconverged infrastructure solutions, uses Apache Pulsar for message streaming, queuing, event sourcing, and general pub-sub use cases. The company chose Pulsar over Kafka due to its robust features and active community support. As of now, Nutanix has 1,700 topics in production with a single producer producing 0.2 million messages per minute and a consumer consuming up to 0.7 million messages per minute. The company plans to use additional Pulsar features like functions, transactions, and delayed messages in the future. One suggestion for improvement is to have more community-based QA teams or forums to ensure feature stability and test coverage.

Season 2 of the podcast "Inspired Execution" has concluded with host Chet Kapoor recapping his interviews with technology leaders from various global enterprises, discussing challenges during the pandemic and providing advice for younger professionals. The season featured conversations with leaders from companies such as Cisco, Goldman Sachs, and PayPal, covering topics like simplicity in product design, working for large organizations versus startups, and Microsoft's approach to a global customer base. Season 3 is expected to continue exploring insights from impactful industry leaders.

Apache Kafka, Apache Cassandra, and Apache Spark Streaming are powerful technologies that can be used together in an architecture to tackle various use cases. Kafka is a massively scalable pub/sub messaging system designed for modern event streaming, while Cassandra is a distributed database management system. Spark Streaming can help enrich data, making it more meaningful and useful. The combination of these technologies allows for efficient movement of data out of legacy systems, handling microservices meshes, reducing the costliness of reads, and managing log events effectively. By using Kafka, Cassandra, and Spark Streaming together, organizations can provide better customer experiences and improve their overall data infrastructure.