July 2024 Summaries

Movies++ is a superior movie search experience built using a combination of technologies including DataStax, Langflow, OpenAI, and TMDB. The process involves creating a database with Astra DB, loading movies into the collection, and then querying the data using Langflow to build a RAG pipeline. This pipeline can be exposed over a web API and consumed in an application. To enhance user experience, GenAI capabilities are added to the UI using Vercel AI SDK, allowing for richer elements such as movie covers and trailers. The full demo is available on GitHub and showcases how DataStax's world-class tooling can be used for building innovative GenAI applications.

In response to growing concerns about infrastructure management as generative AI (GenAI) moves from internal experiments to production-grade applications, DataStax customers have turned to infrastructure automation tools like HashiCorp's Terraform. Now, a new module called the DataStax AI Terraform Module has been developed in partnership with HashiCorp to simplify provisioning, installation, and configuration of various DataStax generative AI solutions and tools such as Astra DB, Langflow, and the Astra Assistants API. This module enables developers to install Langflow locally for GenAI projects and create new Astra DB Vector databases. It also helps operators manage the Astra DB lifecycle, organizations, users, and configure security settings. The AI Terraform Module can be added to existing automation scripts by both new and experienced Terraform users, allowing them easy access to DataStax's powerful generative AI stack. To learn more and try out the module, visit the Terraform GitHub repo.

DataStax has integrated NVIDIA's NeMo Retriever NIM inference microservices into its DataStax Astra DB, a hybrid NoSQL and vector database solution powered by Apache Cassandra®. This integration enhances search capabilities, improves efficiency with accuracy, and supports natural language queries for modern generative AI applications. The latest NVIDIA NeMo Retriever NIM comes pre-installed as the default embedding service in Astra DB, simplifying embedding configuration and generation. Additionally, DataStax hosts the NeMo Retriever microservice within the Astra infrastructure and makes it available to AWS free-tier users. The integration of NVIDIA's NeMo Guardrails ensures safety while maintaining fast RAG applications.

Generative AI (GenAI) has opened up new use cases for developers, including intelligent agents, content creation experiences, synthetic data, language translation, and more. Underpinning these applications are vector embeddings, which allow developers to operate on unstructured data like natural language queries. Unstructured data represents everything from documents to videos and audio files, and GenAI apps rely heavily on it. Vector embeddings represent data in a multidimensional space where semantically similar pieces of content reside close together. Machine learning models are used to convert unstructured data into vector embeddings, with popular providers like OpenAI offering various text embedding models. Selecting the right model is crucial for building successful GenAI apps, considering factors such as relevance, language support, domain specificity, latency, and cost. Once vector embeddings are generated, they should be stored and managed in a vector database designed to handle high-dimensional vectors efficiently. Astra Vectorize simplifies the process by enabling developers to perform CRUD operations directly with unstructured data without having to manually build and maintain intermediate data structures like vectors.

In this article, we explore the use of ColBERT, an alternative method for improving retrieval in Retrieval-Augmented Generation (RAG) applications. Unlike traditional methods that turn a passage into a single vector, ColBERT uses Google's open source BERT model to create vectors for each token in a piece of text. This approach captures better context for terms not part of the training data and overcomes issues with chunking strategies. However, it requires more storage capacity and may result in increased latency compared to regular vector search. ColBERT is available in Astra DB through both LangChain and LlamaIndex, making it a viable option for improving accuracy and relevance in RAG systems.

The adoption of generative AI (GenAI) is rapidly growing, with over half of CEOs evaluating or testing its applications. However, most projects are still experimental and require an architectural approach to enhance relevance, accelerate time to value, and ensure security and compliance. DataStax on Microsoft Azure offers a one-stop generative AI stack and data APIs, dramatically accelerating retrieval-augmented generation (RAG) application development. It supports both structured and vector data, is deeply integrated with preferred tools like Azure Functions, Power Query, and GitHub Copilot, and delivers 74x faster response time, higher relevance, and lower TCO than competitive solutions. DataStax on Azure helps organizations across industries realize efficiency and productivity gains, improve service delivery, and unlock unprecedented value for their customers.

Netherlands-based startup TalentList uses advanced vector search technology to improve candidate matching in short-term staffing, enabling employers to find the best-fit candidates quickly. The platform integrates Bubble.io for no-code app development and Astra DB on AWS for robust database infrastructure and sophisticated search algorithms. By implementing hybrid search, keyword-based search, and vector-based search, TalentList significantly improves efficiency and accuracy in recruitment processes. The company plans to introduce a chatbot using retrieval-augmented generation (RAG) technology and a feedback and review system powered by large language models (LLMs), further enhancing the staffing experience for employers and candidates alike.

Cloud native technology has become mainstream in infrastructure, with Kubernetes being widely used and supported by cloud providers. DataStax has been at the forefront of this movement, launching products like K8ssandra and Astra DB that run on Kubernetes and are fully cloud native. In response to customer demand for a self-managed offering, DataStax has released Hyper-Converged Database (HCD), which includes Mission Control for managing deployment and operations while integrating with existing Kubernetes installations. HCD supports Cassandra workloads and offers notable enhancements such as the Data API and an add-on for the vector search engine JVector. The release of HCD 1.0 marks a future-proof solution that will keep users ahead of the curve, offering features like streaming, OpenSearch integration, and agility in development.

The text discusses the challenges of extracting fine-grained knowledge graphs from unstructured content using large language models (LLMs), which is time-consuming and error-prone. It proposes an alternative approach, a coarse-grained content knowledge graph focused on relationships between content rather than specific concepts or entities. This approach leverages the benefits of vector search and aims to make construction as easy as chunking and embedding the content while preserving the original content until the LLM knows the question to be answered. The main benefits of this approach compared to fine-grained knowledge graphs are that it is lossless, hands-off, and scalable. The text also provides an example of how to build a coarse-grained graph using existing tools and techniques.

DataStax has released the latest version of its distributed database, DataStax Enterprise 6.9 (DSE 6.9), which includes new features and improvements to help modernize enterprise workloads. Key updates include Storage Attached Indexes (SAI) for enhanced performance in numeric range queries, support for Java 11, access via the DataStax Data API, and native vector data type support for building AI applications. DSE 6.9 also offers an easier installation process using DataStax Mission Control for Kubernetes deployments.