April 2019 Summaries

In the ebook excerpt from Streaming Systems, authors thoroughly cover the basics of streaming systems, including their definition, role in data processing lifecycle, and major concerns when working with streaming data. The authors also describe three key streaming technologies: Apache Kafka, Apache Spark, and SQL, highlighting how SingleStore works well with each of them for rapid ingest, complex transformations, and robust SQL support. By reading the excerpt and full ebook, developers can learn how to make streaming part of all their projects and achieve their goals with SingleStore.

Operational analytics is a new database workload that requires an operational SLA, enabling real-time dashboards, predictive analytics, machine learning, and enhanced customer experiences. It demands a new breed of database software that goes beyond legacy architecture, addressing the need for speed, scalability, availability, reliability, and security to support high concurrency, fast data ingestion, and complex queries. Operational analytics workloads require an SLA on how fast data needs to be available, often measured in seconds or minutes, and must handle large datasets, multiple data sources, and disparate formats. SingleStore is a converged system that supports ANSI SQL, has a shared-nothing architecture, and offers features such as transactions, high availability, self-healing, online operations, and robust security, making it an ideal solution for operational analytics use cases like portfolio analytics, predictive maintenance, and personalization.

Apache Kafka and SingleStore together enable the creation of intelligent, real-time applications by providing a publish-subscribe messaging queue that is distributed and durable, serving as a source of truth for data across an organization. The challenges faced by enterprise IT include slow data loading, lengthy query execution, and limited user access, which can be addressed through the use of Kafka and SingleStore's scalable and SQL-architected solution. SingleStore offers cloud-native deployment options, a fully distributed system, and integrates tightly with Kafka to support exactly-once semantics for real-time data pipelines, allowing for live loading of data and meeting tight SLAs for responsiveness.

With SingleStoreDB Self-Managed 6.7, customers can sign up for free, using all enterprise capabilities including high availability and security, up to certain limits such as four leaf nodes with up to eight virtual CPUs per node and 32GB of RAM, rowstore tables in-memory, columnstore tables on disk, community support, and a limited database size. Companies like Culture Amp and Nikkei have achieved outstanding results with SingleStore, improving data-driven decision making and real-time analytics by speeds of over 28 times and 13,500 times respectively, while being able to respond to site visitors' activities in real time. The free tier has been positively received, allowing most experimental projects to run easily within a four-leaf-node configuration without professional support, with the option to upgrade to enterprise offering for dedicated support and high availability implementations.

In pre-modern databases, traditional relational databases support SQL but don't scale out, while NoSQL databases scale out but don't support SQL. Online transaction processing (OLTP) emerged as a way to enable database customers to create an interactive experience for users. OLTP systems were designed to handle large amounts of data and transactions, with key requirements including reliability, scalability, and the ability to run complex queries quickly. However, traditional relational databases struggled to scale, leading to the emergence of OLAP systems that could handle analytics workloads more efficiently. NoSQL databases offered scalability, no schema, and big data support, but came at a cost, including lack of SQL support, no transactions, and slow analytics. Modern databases, such as NewSQL systems like SingleStore, are designed to address these challenges and provide a new workload, operational analytics, which requires handling large amounts of data, complex queries, and high scalability.

SingleStore has been designed and developed as a distributed relational database that brings the effectiveness of the relational database model into the new world of the cloud, containers, and other software-defined infrastructure. As a cloud-native database, it is designed from the ground up to take advantage of cloud computing architectures and automated environments, leveraging API-driven provisioning, auto-scaling, and other operational functions. SingleStore's unique internal architecture gives it both scalability inherent to the cloud and support for SQL for transactions and analytics. It has been running in containers for a long time and uses containerized environments for testing. The database provides command-line tools that integrate easily with on-premises deployment tools and cloud deployment mechanisms, making it crucial to its inclusion as cloud-native software. SingleStore is not limited to cloud applications but can be deployed anywhere - in public cloud providers, modernized data centers, and increasingly at the edge. Its flexibility and portability create a capability that hasn't been available before, with features such as container-friendliness, full scalability, Kafka and Spark integration, microservices support, and more. SingleStore's architecture allows for unbeatable performance and effortless scale, especially when paired with elastic cloud infrastructure, making it an ideal choice for customers looking to modernize their data infrastructure.

The article highlights that the most important new skills for business leaders are not data analysis or machine learning, but rather a drive to find key data and make it useful, creating a culture of constant analysis and action, and choosing the right tools and technologies. Business leaders need to develop these skills to build and maintain a successful business in today's insight-driven economy, where data is crucial for competitive advantage. They must encourage collecting and analyzing data, create a culture of inquisitiveness, and empower teams to act on insights to drive value creation. Additionally, they need to lead the decision-making around data infrastructure, shifting from traditional IT mindset to one that prioritizes agility, and ensure that business leaders, including CEOs, support this shift.