June 2021 Summaries

It marks a watershed moment in database history, as SingleStore unveiled the world's only hybrid multi-cloud, unified analytical and transactional database, offering unrivaled speed, simplicity, scale, and flexibility for data-intensive applications. This groundbreaking solution is a key proof point of SingleStore's innovation focus, with the company expanding its growth and launching initiatives to help customers ignite their own innovation through technology. The announcement also highlights SingleStore's commitment to cultivating community innovation, as it celebrates the opening of its first innovation center in Raleigh, North Carolina, which aims to reduce friction in advancing digital transformation and becoming more data-driven.

SingleStore and Redpanda are used together to solve the operational complexity of global logistics by scaling worldwide parcel shipping to unprecedented volumes. This is achieved through a reference architecture that combines SingleStore's relational database with Redpanda's streaming platform, allowing for efficient processing and storage of large datasets. The system uses a scale-out simulation cluster to generate thousands of packages, emits events into Redpanda topics, and loads them into SingleStore using Pipelines in batches of up to 100 million records. A data model is used to efficiently process and store the data, including two columnstore tables for raw package and transition records, a rowstore table for undelivered packages, and a reference table for locations. The system also includes a stored procedure to keep track of transitions and states in sync with the latest data loaded from Redpanda topics. Data visualization is achieved through Grafana dashboards that showcase performance statistics and actual data, including maps showcasing real-time inventory and current package destinations. The system has been deployed on Google Cloud and achieved a raw computational performance of almost 400k deliveries per second, exceeding the goal of handling 100x the volume of global package delivery per year.

SingleStore's latest release, version 7.5, is a pivotal moment in the database industry, offering a completely new approach to data management for ultra-fast ingest, low latency, high concurrency, and fast analytics capabilities. The release combines separation of storage and compute with system-of-record into a single platform, enabling both transactional and analytical workloads in a hybrid multi-cloud environment. This allows users to effortlessly scale compute while managing storage needs independently, without sacrificing performance or query latency. SingleStore also introduces Point-in-time-Recovery (PITR) for automatic continuous backups, making data safe and protected at multiple levels. Additionally, the release includes Universal Storage technology that supports analytical and transactional workloads in a single table, eliminating design tradeoffs for developers. Other enhancements include lock-free backup operations, new progress reporting capabilities, support for UTF8 4-byte characters, temporary stored procedures, JSON functions, and improved interpreter speed. Overall, this release delivers significant advances in supporting data-intensive applications on SingleStoreDB Self-Managed.

SingleStore, a unified cloud database, has announced its industry-first separation of storage and compute architecture, allowing customers to run operational and analytical workloads simultaneously while flexibly scaling resources. This unique architecture enables users to scale compute independently from storage needs, avoiding high latencies and network bandwidth issues associated with other databases. SingleStore also introduces the ability to suspend and resume workloads effortlessly, making it ideal for mission-critical applications and data-intensive workloads. The company has updated its billing model to offer flexible compute credits and a low per GB rate for storage, minimizing Total Cost of Ownership (TCO). Additionally, SingleStore Universal Storage provides support for multi-column keys, making it easier to design schemas without tradeoffs between analytical and transactional workloads. The database also features new usability improvements, including support for UTF8 4-byte characters, temporary stored procedures, JSON functions, and enhanced interpreter speedup.

Universal Storage Episode 4` concludes a four-part series on SingleStore's patented Universal Storage feature, which allows for analytical and transactional workloads to be handled by a single table type. The latest release, version 7.5, introduces multi-column key support and full upsert capabilities, making columnstore the default table type. This enables applications to reserve rowstores for high-performance OLTP-style operations while using columnstores for most other use cases, reducing Total Cost of Ownership (TCO) and simplifying application development. The feature improves analytics performance on large tables when combined with upserts, fast lookups, and other OLTP-style operations. SingleStore's Universal Storage is designed to support multi-column unique keys, enabling direct upsert workloads into columnstore tables without the need for migration from rowstores or complex data combination operations. Future releases are expected to introduce an in-memory buffer cache and SELECT FOR UPDATE support for columnstores. With Universal Storage, developers can reserve rowstores for high-performance OLTP-style operations while using columnstores for most other use cases, reducing TCO and simplifying application development.

In a recent partnership announcement, MindsDB and SingleStore have integrated their technologies to advance machine learning innovation within the SingleStore database. SingleStore's high-performance capabilities and features such as column stats, histograms, and windowing functions significantly enhance training performance for machine learning models, reducing computational requirements and enabling faster training times. The integration also enables real-time inference and supports various use cases including anomaly detection, handling large, multivariate time series data, and machine learning on data streams. With the partnership, MindsDB users can leverage SingleStore's features to accelerate their machine learning capabilities directly in their database, resulting in top performance for real-time machine learning applications. The collaboration includes a joint webinar and upcoming events to explore joint capabilities, use cases, and best practices.

Pride Month serves as an opportunity to accept and celebrate love in all its forms while also acknowledging the resilience of the LGBTQ+ community. The importance of challenging the status quo and working towards a more loving and inclusive society is emphasized, with social responsibility based on equality, love, and tolerance. In the workplace, it's essential to speak up against unacceptable behavior from a place of love, but ultimately, love itself is worth fighting for, regardless of external circumstances.

Modern SaaS applications require responsive, data-driven user experiences, necessitating efficient analytic queries. SingleStore's new eBook shares the success stories of three developers who leveraged its capabilities to enhance their analytics and achieve faster customer experiences. The eBook provides unfiltered commentary from the developers on their journey to SingleStore, highlighting its benefits such as real-time data analytics, interactive data visualization, and improved scalability. By adopting SingleStore, these developers were able to reduce query performance times by up to three times, improve export speeds, and eliminate cron jobs, ultimately leading to faster, better, and cheaper solutions for their applications.

SingleStore's chief product officer, Jordan Tigani, discussed the need for speed in financial services at a recent DATAcated virtual event. He highlighted how SingleStore is uniquely positioned to execute data-intensive tasks at almost impossible-to-imagine speeds. This is particularly important in applications such as real-time fraud detection, portfolio management, and building rich customer experiences that provide real-time recommendations. Data-intensive applications require fast, reliable access to data for making decisions, with latency being a critical factor. Jordan explained how SingleStore's in-memory database can process transactions at sub-millisecond speeds, scale to handle myriad concurrent users, and use resources efficiently. This makes it an ideal solution for large ride-sharing companies, top-tier banks, and major internet companies that require fast data analysis and reporting capabilities.

We are hiring frontend-focused software engineers who will work on the control plane UI for SingleStore's cloud service, which includes a SQL IDE, Query Plan visualizer, activity monitoring dashboard, and other administrative features. Our technology stack is React/TypeScript, with a backend written in Go and a frontend that talks to the backend via GraphQL. We prioritize full-stack engineering and foster a culture of collaboration between frontend and backend engineers. The interview process involves an application review by our hiring manager, followed by two one-hour technical exercises, and then four interviews covering UI and User Experience, network requests, data structures, and team fit. We also conduct a roundtable discussion to align on the candidate's performance and cultural fit. Our goal is to have a fast yet thorough process that provides valuable information about candidates, and we're constantly iterating and seeking feedback to improve our interview loop.

Modern applications live in the cloud and access and generate large amounts of data, requiring a modern data infrastructure that can scale seamlessly as user numbers grow. Data-intensive applications use various types of data, including user data, 3rd party data, telemetry data, and aggregate data, to provide insights and drive business decisions. These applications require a consistent, highly available, durable, and resilient data infrastructure that can handle the volume, velocity, and variety of data. The modern approach to data infrastructure emphasizes scalability, security, privacy, and semantic understanding of data, with a focus on delivering real-time and instant access to users. To meet these needs, modern applications often rely on specialized database systems that support natively loading data from various formats and delivering analytics without noticeable lag.

The author believes that we are living in a fascinating era of technology where data is playing an increasingly important role, and every generation will make this claim. The term Data Science emerged in the early 2000s as big data was on the rise, with digitally native companies building businesses around new data sources, changing the focus of computer science from computation to data. Fast forward to today, a new form of artificial intelligence has augmented human capabilities over the past decade, with data becoming a way of life, thinking, reasoning, understanding, and communicating. Data alone do not change the world; it is through decisions based on data that we affect the world around us. The author proposes building people-literate technology, where technology understands what we need and delivers that for us, with the traditional role of the CIO fading and the Chief Data & Analytics Officer becoming the true information officer in the organization. The contribution and responsibility of data scientists will be elevated to something essential: running the business with data.