September 2024 Summaries

ColBERT Live! is a production-quality library for semantic search with an off-the-shelf vector database. It addresses the limitations of Stanford ColBERT by supporting updates to indexed data and integration with other application data. The library uses the Answer AI colbert-small-v1 model, which provides faster embeddings generation and more relevant results compared to the original model. ColBERT Live! also introduces new techniques for optimizing search relevance while reducing overhead. It is designed for robust, production-ready semantic search with out-of-domain search terms and can integrate with existing vector databases.

The RAG++ AI Event tour made its third stop in London, gathering over 200 developers, CxOs, and founders to share knowledge and learn from various industry experts. DataStax CEO Chet Kapoor announced that the company has been named a Leader in Forrester's Wave™ for Vector Databases, Q3 2024. A fireside chat between Kapoor and NBCUniversal CTO Patrick Miceli discussed AI usage in media and entertainment companies. Breakout sessions covered topics such as generative AI in regulated industries and partnership news, including a new DataStax Langflow Glean Component that enables seamless integration with the Glean Work AI platform for more informed decision-making processes.

In this article, four methods to generate vector embeddings in Node.js are discussed: locally, via API, via a framework, and with Astra DB's Vectorize. The author explains how to use Transformers.js to create vector embeddings using local models like all-MiniLM-L6-v2. They also explore the use of APIs from providers such as OpenAI, Google, Cohere, Voyage AI, and Jina for generating embeddings. Additionally, they introduce two frameworks, LangChain and LlamaIndex, that provide abstractions over various parts of the GenAI toolchain, including embeddings. Finally, the author presents Astra DB's Vectorize feature, which allows users to store vectors in a vector database like Astra DB and perform searches against them.

DataStax has been named a Leader in Forrester's inaugural Wave™ for Vector Databases, being the only hybrid vector database Leader. This recognition highlights their commitment to providing developers and companies with an AI platform that enables them to build AI applications quickly. Unlike other players in the market, DataStax offers a single data platform that combines various types of data, reducing complexity and decision fatigue. With over a decade of experience handling high-scale NoSQL production workloads, DataStax aims to empower organizations to build and scale AI-driven applications. They thank Forrester for this recognition and their customers for their continued trust and partnership.

Glean and DataStax have partnered to enable developers to build generative AI applications more easily and efficiently. The collaboration allows developers to combine the best-in-class tools of both platforms, harnessing advanced search capabilities and robust database solutions. The partnership includes the development of a new tool, the DataStax Langflow Glean Component, which enhances Langflow agent flows by integrating Glean's powerful retrieval capabilities into their implementations. This collaboration streamlines the development process for GenAI applications, offering seamless interoperability between DataStax and Glean solutions.

Valeri Karpov, the creator of Mongoose, demonstrates how to build a Notion clone using Astra DB and Mongoose in this insightful blog post. The process involves adapting notion-clone to use stargate-mongoose (only five lines of code) and deploying it on Vercel. Key points include upgrading Mongoose version, installing stargate-mongoose, setting up an Astra DB account and database, porting backend logic into the frontend folder, and refactoring getServerSideProps() functions to call controller functions directly. The source code is available on GitHub, and a live example can be tried out on Vercel.

Fiddler AI and DataStax have collaborated to enhance retrieval-augmented generation (RAG) based large language model (LLM) applications by integrating Fiddler's AI Observability platform with DataStax's Astra DB. This integration aims to optimize the performance, accuracy, safety, and privacy of GenAI applications by utilizing Astra DB's real-time vector search capabilities and Fiddler's LLM application scoring powered by Trust Models. The technical process involves onboarding applications like DataStax's WikiChat to the Fiddler environment, allowing for the monitoring of prompts and responses across various trust-related dimensions such as faithfulness and PII leakage. Developers can use tools like Next.js to ingest data into Fiddler, enabling real-time detection and diagnostics of issues such as prompt injection attacks and toxic responses. By tracking relevant LLM metrics, developers can align these insights with business KPIs and explore further enhancements through resources like Fiddler's AI Chatbot guide and Astra DB's free offerings.

Retrieval-augmented generation (RAG) systems sometimes fail to provide detailed and accurate responses due to limitations in retrieving information from deep knowledge bases. Graph RAG, which involves augmenting the RAG system with a knowledge graph for retrieval, can help address this issue by enabling deeper exploration of data sets and providing more precise connections between documents. Key concepts behind improving RAG performance include using well-defined and meaningful connections such as HTML links, specialized keywords, terms and definitions, and document structure metadata. Graph RAG has proven effective in various domains like technical documentation, legal contracts, and large wikis or knowledge bases.

Retrieval-augmented generation (RAG) applications require text data to be split into smaller chunks and prepared for use in a vector database like Astra DB. This process, called text chunking, is crucial for improving retrieval accuracy and creating more accurate and useful RAG systems. Several libraries are available in JavaScript for text chunking, including llm-chunk, LangChain, LlamaIndex, semantic-chunking, and the Unstructured API. Each library offers different features and capabilities, allowing developers to experiment with various options to find the best fit for their specific needs.

The new feature, Graph Vector Store, developed in collaboration with the LangChain team, enhances retrieval-augmented generation (RAG) applications by combining semantic similarity and knowledge graphs. This hybrid approach improves data retrieval accuracy and completeness. By overlaying graph connections onto existing vector databases, users can benefit from both vector similarity and knowledge graph connectivity. Graph Vector Store is a drop-in enhancement to traditional RAG systems and offers a more robust solution for data retrieval, ensuring that relevant information is not overlooked.

Skypoint, a provider of data, analytics, and AI services to healthcare providers, has developed SherloQ, a highly accurate Text2SQL engine built on Astra DB. The specialized approach of SherloQ delivers unmatched accuracy and cost-efficiency, making it the tool of choice for organizations that demand more from their data. In benchmarks on production workloads, SherloQ achieved a 92% accuracy rate in converting natural language into SQL queries, significantly improving over general-purpose AI tools like GPT-3.5 and Snowflake's Cortex Analyst. SherloQ is designed to understand the unique terminologies, data structures, and nuances of industries that operate under strict regulatory frameworks, making it particularly valuable in healthcare, public sector, and financial services. The tool offers seamless integration with existing data infrastructures, advanced error handling, state-management capabilities, and robust security features. SherloQ's combination of industry-specific accuracy, cost-effectiveness, and advanced features sets a new standard in Text2SQL technology.

To become an AI power user, one must effectively leverage AI tools while understanding their limitations and optimal use cases. LLMs are powerful but resource-intensive and unpredictable, so using them judiciously as part of a larger system is key. The "Goldilocks Zone" of LLM usage varies depending on the specific use case. Understanding the boundaries and limitations of LLMs is crucial for effective use. Power users often employ strategies such as using small, focused models and avoiding magical thinking. Building and deploying a chatbot into production can be an excellent starting point for learning about AI applications and practical uses.

This post discusses an application built using Astra DB and Langflow, which turns any website into a fun multiple-choice quiz with the help of generative AI. The technique used is retrieval-augmented generation (RAG), which provides real-time context to large language models. To create this application, users can visit DataStax Langflow, import the provided flow JSON file, and then make it available over the internet via API. This end-to-end pipeline demonstrates how Generative AI can be used to democratize AI for all developers. The GitHub repo containing the Langflow JSON file and a user interface connected to it is also mentioned for further exploration.

On Wednesday, DataStax held its latest RAG++ AI Event in New York City with over 500 developers, CXOs, founders, and others attending. The event focused on the challenges faced by developers when working with generative AI applications and featured speakers from IBM, Glean, AWS, NVIDIA, Google Cloud, Unstructured, and more. DataStax CEO Chet Kapoor discussed the company's work in building a stack that provides developers with an efficient path to production for AI development lifecycle. Industry leaders participated in panel discussions on improving AI-driven systems, AI strategies in healthcare and financial services industries, and new integrations aimed at making it easier for developers to get their GenAI applications to production quickly. DataStax also announced several new partnerships and updates, including the Langflow API, Unstructured document ingestion capabilities, and a new integration with Glean. The next RAG++ AI Event will be held in London on September 24th.

Apache Cassandra 5.0 is a significant upgrade with new features and improvements that enhance usability and capabilities for the world's most powerful distributed database. The release includes Storage Attached Index (SAI), Trie indexes, support for JDK17, Unified Compaction Strategy (UCS), and vector search capabilities. These advancements provide increased performance, efficiency, and flexibility for both operators running large clusters and developers building applications. Cassandra 5.0 marks the end-of-life for Cassandra 3.x, prompting organizations to plan their upgrade strategy. The diverse contributions from various contributors make this release truly special, reflecting the collaborative nature of open-source development. DataStax is committed to supporting users on their Cassandra journey with a range of options and services tailored to individual needs.

Langflow has made it easier for developers to experiment with generative AI by offering a hosted API platform on the DataStax platform, available in public preview. This eliminates the need for self-hosting and infrastructure management, allowing users to generate an API key and deploy their apps much faster. The process of building, deploying, and using an API for a GenAI project powered by DataStax Langflow can be completed in less than five minutes. With this new feature, developers can quickly create chatbots, recommendation engines, and AI agents for their applications.

Unstructured.io has partnered with DataStax to enhance its Astra Data Loader and introduce document ingestion capabilities in the low-code IDE Langflow. The updated Astra Data Loader now supports PDF files, streamlining the process of ingesting unstructured data. Additionally, a new Unstructured component has been introduced for developers using self-managed Langflow, enabling advanced document processing with flexibility and ease. These integrations simplify data preparation and ingestion for generative AI applications, allowing developers to focus on creating innovative, intelligent applications that drive business value.

Melissa Herrera is a Developer Relations Engineer at DataStax with a background in Computer Science and databases. She has been with the company for three years, transitioning from support engineer intern to her current role. Her interest in DevRel began when she assisted the DevRel team in creating content. Melissa's recent work on Fashion Buddy, an AI-powered app that helps users find similar clothing items, led her to become more involved in DevRel projects and attend conferences and events. As a developer relations engineer, she is now immersing herself in generative AI (GenAI) and working to inspire developers to explore the world of AI. She can be found on LinkedIn or through DataStax's Developer Discord.

The text discusses the author's interest in universal translators, inspired by science fiction works like "The Hitchhiker's Guide to the Galaxy." It then delves into their exploration of creating a real-time language translation app using Generative AI (GenAI) and Langflow. They experimented with different LLMs and prompt templates to achieve desired results, highlighting the ease of use and flexibility provided by Langflow in this process. The author plans to continue building the app itself in future parts of the series, focusing on user interface design and real-time audio processing.