July 2017 Summaries

The text discusses the challenges of testing a distributed database like SingleStore and how Docker helped alleviate those issues. The company started with a DIY approach to testing their software, using a cluster of bare metal boxes and later an OpenStack cluster, but eventually found that containers provided a better solution for scaling and isolation. They built a custom scheduling system and key value storage system called Pokedex to support their containerized environment, which has improved the efficiency of their testing process. The experience with Docker has been highly beneficial, allowing SingleStore to test their software at scale and maintain reliability. However, there are still challenges related to Linux namespace abstraction and debugging in containers that need to be addressed.

At SingleStore, developers were trying to solve a problem with obtaining the thread group identifier (`tgid`) using the `SYS_tgkill` Linux system call. They initially used a simple prototype that read from `/proc/self/status`, but it broke when newer Linux distributions added a new field to this file. The team investigated and discovered that the `tgid` was actually the same as the process ID (`pid`). By using the `getpid` system call directly, they were able to stabilize the tool and simplify the code, making it work on all environments tested.

The Amazon Web Services Summit will take place in Chicago from July 26-27, 2017, and is a free event designed to educate attendees about the AWS platform and help develop skills for designing, deploying, and operating infrastructure and applications. SingleStore will be exhibiting at the HUB, Partner & Solutions Expo, where attendees can view a demo and speak with subject matter experts. The summit will offer breakout sessions led by AWS experts and top customers, covering technical content, customer stories, and new launch announcements. Some notable breakout sessions include "Real-time Dashboards with Kinesis Analytics & Amazon Rekognition", "Secure Your Cloud Investment: Mastering AWS Identity Access Management (IAM)", "Transforming Your Business with Analytics on the Cloud", "Cloudreach: Building an Effective Cloud Operating Model on AWS", and "Real-Time Streaming Analysis & Visualization using Apache Flink on Amazon EMR and Kibana". These sessions are geared towards keeping attendees informed on technical content, customer stories, and new launch announcements.

Google Analytics' launch was a disaster due to its high hardware requirements, but it eventually became a highly profitable product. The key to Google's success lay in allowing customers to segment and calculate flexible aggregates using raw unaggregated data. Most companies have built analytics with precomputed data due to cost and technology constraints. SingleStore is changing this conventional wisdom by offering companies the ability to serve raw unaggregated data performance to a range of customers. This is achieved through scale-out, columnstore query execution, and efficient data isolation per customer. Scale-out allows for distributed systems with dedicated compute power to meet demand, while columnstore query execution enables simple queries to run at hundreds of millions to over a billion data points per second per core. Efficient data isolation per customer ensures that only scanned data belongs to a single customer, allowing high concurrency and fast response times. SingleStore's innovations in query processing capabilities enable sub-second performance for complex queries on large datasets.

A cloud data warehouse is essential for organizations dealing with rapid data growth and diverse business needs. To select the best cloud data warehouse, it's crucial to identify your use case and consider factors such as ad-hoc analysis capabilities, machine learning support, real-time analytics, mixed workload support, usability, performance, cost, flexibility, scalability, and security. A robust data warehouse should deliver fast consistent responses across various query types, provide native interoperability with data preparation and statistical tooling, offer rapid sandbox configuration for quick experimentation, and ensure comprehensive security to shield users from sensitive data. Additionally, a cloud data warehouse should support real-time streaming, scalable concurrency, and exactly once semantics for real-time data de-duplication and enrichment. By understanding these capabilities, organizations can make an informed decision when evaluating their next cloud data warehouse investment.