August 2023 Summaries

There is a lesson in the story of London's thriving black cabs, which have survived disruption from ridesharing by adapting to technology while maintaining their human touch, including extensive training and deep knowledge of the city. The experience offered by traditional taxi services can provide a way for businesses to withstand new technologies through superior customer experiences. In contrast, rideshare companies with basic GPS systems lack the personal touch that black cabs have managed to preserve. London's black cab industry is an example of how competition can drive innovation and resilience, highlighting the importance of staying adaptable in a rapidly changing world.

This guide provides a comprehensive walkthrough on creating embeddings at scale using OpenAI's text-embedding-ada-002 model and SingleStoreDB. It covers the integration of the OpenAI model, configuration of connections to SingleStoreDB, writing the core functionality, packaging files for AWS Lambda, and scheduling the function with Amazon EventBridge. The guide aims to provide a practical approach to working with textual data at scale, enabling users to create embeddings efficiently and securely.

Google Vertex AI and SingleStore are being used to build enterprise-grade private Large Language Models (LLMs) applications that are data-aware and custom to a company's needs. Companies can now use Google Vertex AI APIs to choose from available models, fine-tune them with company-specific requirements, and expose them as APIs. They can also use SingleStore database deployed on Google Cloud Platform to store SQL and JSON data, run analytics in split seconds, and do both lexical and semantic search to curate the most relevant and fresh data for LLMs. This enables companies to take full advantage of current advancements in generative AI by storing and querying vector data along with other types of data using SQL in real-time, using Notebooks to chain multiple LLMs, and having connectors and pipelines that enable fast ingest of data from different sources.

SingleStore has appointed Nadeem Asghar as Senior Vice President of Product Management & Strategy, bringing over 25 years of experience in cloud, data, streaming and AI platforms. Prior to joining SingleStore, Nadeem was at Cloudera, where he drove capabilities for big data and cloud initiatives. He's excited to work on SingleStoreDB's hybrid architecture, generative AI, and data mesh applications, aiming to simplify customer architectures and reduce total cost of ownership. With a focus on innovation and solving real business problems, Nadeem emphasizes the importance of effective communication, adaptability, and continuous learning in product management. As he joins SingleStore, he looks forward to working with internal teams and customers to take the product to the next level and meet future customer needs.

SingleStoreDB's Universal Storage is designed to handle complex queries on large datasets in narrow or wide tables, providing real-time streaming ingest and millisecond response times for web-scale applications. The storage engine's design foundation on speed and complexity ensures app performance, with indexing and query optimization engines working together to guarantee low latency. Key considerations for optimizing schema include shard and sort key selection carefully to maximize query speed, using hash indexes to speed up equality lookups, and adjusting columnstore_segment_rows and Columnstore_flush_bytes to facilitate wider rows loading through memory. SingleStoreDB also addresses traditional pain points of columnstores with the addition of a columngroup index, which improves transactional performance by up to 6x. Query compilation and plancache improve query performance, but may consume additional memory and disk resources. Sizing a wide-table use case requires considering uncompressed data size, compression ratio, CPU:partition ratio, and concurrency requirements, with the ability to scale-out as needed to meet high concurrency queries and large ingest volumes.

IBM Data Replication, a leading Change Data Capture (CDC) tool, is being integrated with SingleStoreDB to power real-time use cases. SingleStoreDB is a distributed, relational database that offers speed and scalability for transactional and analytical workloads, while also providing enterprise security protocols, a data fabric architecture, and real-time synchronization capabilities. This integration enables organizations to build real-time analytical solutions by moving data from supported sources directly into SingleStoreDB, eliminating latency issues and enabling secure, real-time data movement. With this partnership, organizations can drive business growth by building game-changing real-time analytical applications that leverage the power of both IBM Data Replication and SingleStoreDB.

With the integration of Singlestore Helios and Datadog, customers can now harness advanced monitoring capabilities to unlock real-time insights and comprehensive metrics on their Singlestore Helios instances. This powerful collaboration promises to revolutionize how organizations monitor and manage their database infrastructure, providing a robust solution that enhances monitoring capabilities, offers powerful analytics and alerting features, and provides a unified platform for monitoring the entire IT stack from a single location. With this integration, customers can gain real-time insights into key performance metrics, troubleshoot performance issues proactively, and optimize their Singlestore Helios instances for peak efficiency, leading to improved productivity, faster insights, and better decision making.