April 2019 Summaries

This blog post discusses advanced options for using DataStax Bulk Loader (dsbulk) to load, unload, and count data in a DSE cluster. It covers specifying host IP or name with the -h parameter, changing the port number with the -port parameter, handling Kerberos-enabled clusters, setting consistency levels with the -cl parameter, controlling error behavior with the -maxErrors and -logDir parameters, adjusting logging verbosity with the --log.verbosity parameter, monitoring progress with the -reportRate parameter, and disabling JMX metrics with the --monitoring.jmx parameter. The post also provides links to related blog posts and resources for downloading dsbulk.

The talk "Get Better Operation Insights on DataStax Enterprise Clusters" by J.B. Langston and Manikandan Srinivasan at the DataStax Accelerate conference will focus on scaling DSE monitoring using features like Metrics Collector, sharing insights from their work with large enterprises and announcing future solutions to aid operators. The conference also offers numerous operator-oriented talks and opportunities for networking and interaction among speakers and attendees.

DataStax Accelerate is an upcoming event featuring valuable talks from developers and operators who share their best practices and experiences with Apache Cassandra and DataStax Enterprise. Companies such as Apple, Netflix, Facebook, Yelp, Deloitte, Capital One, Walgreens, and T-Mobile will be presenting at the event. One highly anticipated talk is "How to Become the Lord of the Rings" by Sean Durity from The Home Depot, who has extensive experience in administering Apache Cassandra and DataStax Enterprise at scale. The event takes place at the Gaylord National Resort & Convention Center in Maryland on May 21-23, and registration is open now.

The text discusses the use of Apache Cassandra™ and DataStax Enterprise (DSE) in cloud-based data management, highlighting their benefits and popularity among users. It also mentions that despite the advantages, poorly designed applications can still encounter issues in the cloud, with the cloud now being the second biggest cause of downtime according to a report by the Uptime Institute. The author recommends attending Orzota's session at the Accelerate conference for advice on avoiding common pitfalls when using Cassandra and DSE in the cloud.

The upcoming DataStax Accelerate conference is highly anticipated, with a focus on innovations in the enterprise technology space and digital transformation in today's cloud-dominant market. Exciting presentations from companies like Facebook, Yelp, Netflix, T-Mobile, Deloitte, and more are expected to provide valuable insights into current trends. The session "Opioid360 – How AI Can Help Government Leaders Save Lives and Dollars, 24/7/365" by Deloitte is particularly intriguing as it showcases how DataStax Enterprise (DSE) is being used to address the opioid addiction crisis. The conference promises to be a great opportunity for professionals in the field to network and discuss the latest advancements in enterprise software.

Modern applications need to be scalable, high-performance, resilient, available, and highly secure. To achieve this, organizations should consider using microservices, hybrid cloud environments, modern databases, distributed apps with data replication across geographies, masterless architectures for maximum uptime, and an underlying data layer that offers industry-leading security capabilities in any kind of cloud environment. These features will ensure better user experiences, higher output, and a healthier bottom line.

Artificial Intelligence (AI) and Internet of Things (IoT) have become integral components of our daily lives, impacting mission-critical applications to personal assistants like Alexa and Cortana. Worldwide investments in IoT technology are expected to reach $1.2 trillion by 2022, with the consumer industry leading the way followed by insurance and healthcare. AI is also experiencing massive growth; Gartner reported that the number of companies implementing some form of AI tripled in just a year. Both AI and IoT are inherently interlinked, enhancing and strengthening each other for use cases involving high-velocity data. The combination of AI and IoT will shape corporate data strategies for years to come. A hybrid cloud database is essential to support the future of information technology as it ensures worldwide access to highly responsive data across many geographies, providing speed, reliability, security, reduced costs, and enhanced operational efficiencies.

The DataStax Accelerate Bootcamp is designed for developers looking to expand their knowledge and skills in today's rapidly evolving industry. This hands-on, guided experience covers the fundamentals of working with cutting-edge technologies such as distributed databases like Apache Cassandra. The bootcamp offers valuable learning opportunities for both experienced professionals and beginners alike, taking them from first-launch to building a working video-sharing application. With real-time feedback from instructors and an engaging learning environment, the DataStax Accelerate Bootcamp provides a comprehensive and enjoyable experience for all participants.

In 1991, Stewart Alsop predicted that mainframes would become obsolete by 1996. However, this prediction did not come true as new technical solutions like Apache Cassandra and DataStax Enterprise (DSE) emerged to handle massive processing jobs once dominated by mainframes. Additionally, the dynamics of risk have changed drastically, with the "risk of change" often being less than "the risks of the status quo." To modernize technology stacks effectively, organizations should start now and choose their own pace and scope of architectural modernization. DataStax Accelerate, a premier Apache Cassandra conference, will feature speakers discussing transformational stories and networking opportunities for those interested in mainframe modernization strategies.

DataStax Accelerate, an Apache Cassandra conference, is set for May 21-23 in Washington, D.C. The event features eight tracks catering to developers of all skill levels and roles. Some highlighted sessions include Sergiy Smyrnov's discussion on Walgreens' use of DataStax and Apache Cassandra in Microsoft Azure Cloud, Pascal Desmarets' case for reinventing data modeling for agile development, Andrew Prudhomme and Abrar Sheikh's presentation on Yelp's Data Pipeline, Mike Treadway's insights on running a globally distributed Cassandra cluster in Kubernetes, and Troy Motte's migration of billions of records from MySQL to Apache Cassandra. The full schedule is available online, with registration encouraged for those interested in attending.

The text provides a guide on how to install Apache Cassandra, Apache Spark, and their respective connectors on one node or local instance for machine learning experimentation. It outlines the steps to download, install, and configure each component, as well as testing the connection between them using PySpark and Jupyter Notebooks. The guide also provides references to additional resources and sample notebooks for further exploration of data with these tools.

The text discusses a talk about running data-intensive systems in Kubernetes, covering topics such as pods, jobs, stateful sets, and replica sets. It also mentions custom resource definitions (CRDs), operators, and HELM Charts, which are features that can help host various complex distributed systems. The speaker is @Lenadroid.

In April 2018, DataStax Enterprise (DSE) 6 was released with significant performance improvements over previous versions and Cassandra. The new thread-per-core asynchronous architecture and storage engine enhancements contributed to this boost in speed. A year later, Intel introduced the second generation of its Scalable processors, which further increased the performance gap between DSE and Cassandra due to optimizations for Optane™ Persistent Memory Modules. The combined innovations from DataStax and Intel have resulted in a highly efficient database, CPU, and storage system. Benchmark tests using DSE 6.7 and DataStax Distribution of Apache Cassandra® revealed that the new Intel® Optane™ Persistent Memory Module outperforms NVMe drives on a 4-node cluster with 40-core Cascade Lake Xeon® Scalable data center processors. Users are encouraged to upgrade to DSE 6 or 6.7 and explore Intel's latest product line for enhanced performance without data model changes. DataStax is also collaborating with Intel on software innovations related to Cassandra, which will be previewed at the DataStax Accelerate conference in May 2019.

Microservices adoption is expected to reach 90% by 2020, driven by the need for adaptability and scalability in today's rapidly evolving business environment. The main reasons for moving to microservices include increased adaptability to changes, efficient scaling of individual modules, support for digital transformation initiatives, enabling quick adoption of new technologies, and faster time-to-market for applications. Microservices architecture also requires a suitable database, such as multi-model databases or NoSQL data stores, to fully leverage its benefits. Developers are empowered by microservices' flexibility and can build more powerful applications with the right tools and support.

Federal and government agencies face challenges in managing their data amidst increased adoption of cloud technology and mobile devices. The key lies in choosing the right database management solution that ensures data security while driving efficiency and innovation. Three crucial factors to consider are using a NoSQL database instead of an RDBMS, opting for an always-on database over a highly available one, and embracing hybrid or multi-cloud computing environments. By leveraging these technologies, governments and federal agencies can modernize their data strategies and capitalize on the same benefits as private sector organizations in terms of innovation and disruption.

In this blog post, we continue exploring the DataStax Bulk Loader (dsbulk) command for data loading. We discuss how dsbulk allows comments in input files and how to handle null values in data. Additionally, we cover handling string-based NULLs, using different date formats, populating data with current dates or timestamps, working with collection and user-defined types (UDTs), setting TTL and timestamp for the data, performing a dry run before actual loading, and controlling the number of writes per second and outstanding writes at any given time.

Fog computing is a distributed computing framework that extends cloud computing capabilities closer to the end user or devices, enabling local processing of data in smart devices and improving the operation of compute, storage, and networking services between end devices and cloud computing data centers. It supports IoT, 5G, AI, and other applications requiring ultra-low latency, high network bandwidth, resource constraints, and added security. Fog computing is becoming increasingly important as the number of connected devices grows, with predictions that it could reach over $18 billion worldwide by 2022. It offers a solution to challenges faced by cloud computing, such as real-time processing, latency, security, and privacy concerns.

A majority of companies have adopted a hybrid cloud strategy due to its unique benefits. To transition to this model, businesses should evaluate existing workloads and future needs, carefully select the appropriate cloud options, consider compliance ramifications, determine which cloud vendor will be utilized for each application or process, plan data storage and backup solutions, implement a phased migration approach starting with high-value applications, periodically evaluate the strategy's effectiveness, and optimize their data management strategy. The benefits of transitioning to hybrid cloud include improved cost-effectiveness, security, operational efficiencies, and scalability.

The author, a software industry veteran, shares their experience building a Python implementation of the KillrVideo microservice tier. They discuss why they built service scaffolding first before implementing any business logic and how this approach aligns with Test-Driven Development (TDD) principles. The integration tests for KillrVideo are implemented in Java using the Cucumber framework, focusing on the microservices tier. The author highlights the benefits of running individual tests by name to save time during development iterations. They also emphasize the importance of testing services against an existing web application as it complements automated testing and can help discover bugs not covered by integration tests. In the next post, they plan to discuss implementing the actual business logic of the KillrVideo Python services, including data persistence using DataStax Enterprise and the DataStax Python Driver.

Lightweight transactions enable highly durable distributed systems to operate in an ACID-like manner with ease. They ensure that data read is correct and current, which is crucial for maintaining data integrity. In DSE, lightweight transactions are implemented using the Paxos protocol combined with normal read/write operations. However, they have a relatively high cost of use and should only be utilized when necessary, such as in situations where concurrency is a factor. Consistency levels SERIAL and LOCAL_SERIAL are specific to lightweight transactions and provide linearizable consistency for handling failure scenarios.

The text discusses the challenges faced by companies when their outdated database technology fails to handle high transaction volumes or provide real-time data access for millions of customers. It highlights that modern NoSQL technologies like Apache Cassandra™ and DataStax can help overcome these issues, offering scalability, reliability, and support for hybrid and multi-cloud computing environments. The text emphasizes the importance of a robust data architecture to prevent negative customer experiences and maintain brand reputation.

Enterprises are increasingly adopting open source technology due to its accessibility, cost-effectiveness, and ability to attract top talent. However, challenges arise at the enterprise level with open source solutions like Apache Cassandra. These include hidden ongoing maintenance costs, dependency on voluntary contributions from third-party developers, security compliance issues in regulated industries, and a lack of efficient support services. Despite these challenges, partnering with the right organization can help enterprises unlock the full potential of open source technology without any downsides.