September 2021 Summaries

As a developer, you spend time making choices that can impact the performance and scalability of your application. These decisions often depend on the tradeoffs made by programmers at the next layer down the stack. For example, when choosing a database, you may need to consider factors such as horizontal vs vertical scalability, column-oriented vs row-oriented storage, physical storage choices, data loss protection, and query performance. SingleStore's Universal Storage engine is designed to provide a balance between these competing demands, offering features such as high availability, unlimited storage, incremental backup/restore, automated statistics collection, vectorized SIMD execution, and hot-swapping of optimized results during compilation. By understanding the tradeoffs involved in building a database like SingleStore, developers can make informed decisions about their own application's performance and scalability needs.

The Workload Monitoring UI in SingleStore allows users to analyze their clusters' workloads by viewing activities that ran during a specific period, as well as query resource usages and other properties. The feature can be implemented using either real-time recording or historical monitoring, with the latter requiring cluster-level configuration. In both cases, the frontend retrieves data from a cumulative table in the database, calculates the delta between activity groups, and displays results in a graphical and aesthetically pleasing way. The UI also supports browsing resource usage by node and provides additional features such as unit conversions and node breakdowns to enhance user experience. The feature is currently part of SingleStore Studio but will soon be available in SingleStore Helios UI as well.

Oliver Schabenberger, Chief Innovation Officer at SingleStore, sat down with David Yakobovitch to discuss the role data science plays in making breakthrough innovation possible. He shared his passion for data and machine learning, analytics, and data-intensive applications. Oliver explained how data science is evolving and becoming more democratized, allowing more people to work with data and contribute to driving business decisions. He emphasized the importance of connectivity and automation in technology development, as well as culture, highlighting SingleStore's focus on making data-driven decisions more accessible and efficient. The conversation covered topics such as explainable AI, model-driven decision-making, and the future of data science, where Oliver sees a shift towards serving customers better through continuous improvement and innovation.

The modern digital services economy requires businesses to respond to customer requests in real-time and access business insights on a second-by-second basis, driving the creation of massive volumes of data. To keep up with these demands, traditional databases are being wire together, but this approach carries a high cost of maintenance and creates database sprawl. SingleStore's multi-model, multi-cloud database is designed to address these challenges, providing a single database that can handle multiple types of data workloads. The company has seen significant growth, including a 300%+ increase in new customers for its managed service offering and 150% YOY growth in cloud revenue, with investors doubling down on the company's Series F round led by Insight Partners and Hewlett Packard Enterprise. SingleStore's customers include Uber, Thorn, and IEX Cloud, who have seen benefits from using the database to solve critical issues and improve their operations. The company plans to continue innovating and pushing the boundaries of data science to bring more capabilities to its customers, with a focus on supporting builders and enterprise customers.

S2's bottomless design separates storage and compute capabilities, allowing for cost savings, improved performance, and elasticity. It achieves this by committing transactions to the tail of the log on local disk and replicating it to other nodes for durability. Newly committed columnstore files are pushed to blob storage as quickly as possible asynchronously after being committed, while hot data is kept cached locally on disk for use by queries and cold data is removed from local disk. This design has several advantages, including small write transactions having no extra latency compared to S2 that doesn't use blob storage, new replicas and hosts being able to be spun up quickly by pulling data from blob storage, and the blob store acting as an extra layer of durability with point-in-time recovery capabilities. However, it also has some disadvantages, such as durability and availability not being separated in S2 today, and relying on blob storage for cross-region high availability which may result in loss of some data on local disk during a region outage. Overall, the bottomless design allows for improved elasticity at lower costs by storing history in the blob store and using faster local disks for cached data.

The authors integrated Kerberos authentication with SingleStoreDB Self-Managed (MySQL protocol) authentication in a browser-based application, SingleStore Studio. To achieve this, they extended the MySQL "Auth Switch Request Packet" to send an HTTP GET request to a custom API endpoint controlled by Studio's web server when it receives the "auth_gssapi_client" auth request. This endpoint implements only the first part of the SPNEGO protocol, which forces the browser to initialize the Kerberos authentication flow with the SingleStoreDB Self-Managed cluster. The frontend then grabs the service ticket from the response and injects it into the MySQL authentication packet, allowing the SingleStoreDB Self-Managed cluster to validate the service ticket and authenticate the user. This solution allows for seamless integration of Kerberos authentication in a browser-based application without requiring additional desktop distributions or dependencies.