March 2024 Summaries

Microsoft's Semantic Kernel, an open-source SDK, has integrated with DataStax Astra DB to help developers build generative AI agents that can call existing code and leverage retrieval-augmented generation (RAG). This integration enables the creation of production-grade GenAI apps and AI agents using the precision and relevance provided by RAG. Semantic Kernel is a GenAI/RAG application and agent orchestration framework in Microsoft's stack of AI copilots and models, offering capabilities for managing contextual conversations, planners for multi-step functions, and connections for third-party APIs. The Astra DB connector in Python enables Astra DB to function as a vector database within Semantic Kernel, allowing developers to build powerful RAG applications with extended contextual conversation capabilities and multi-function or planner capabilities on a globally scalable vector database. This integration benefits the GenAI developer community, particularly those working in the Microsoft/Azure ecosystem, by enabling more intelligent, responsive, and personalized AI applications that can transform industries.

PTC Computer's vision was to transform their eCommerce platform with an AI-powered search and conversational AI Assistant. They partnered with DataStax to build a seamless omnichannel experience, utilizing Astra Streaming's real-time data pipelines for retrieval of valuable contextual data. The AI Assistant leverages large language models (LLMs) and multi-modal models to provide natural language queries, voice in local languages, and image interaction capabilities. It also employs a well-established architecture pattern named retrieval-augmented generation (RAG). Astra DB's vector search provides hybrid search capabilities, minimizing hallucinations with higher relevance and faster response times for PTC. The AI Assistant collects data to provide personalized experiences and improve efficiency by automating processes.

OpenAI has added streaming support to its Assistants API, improving user engagement and providing reassurance that the system is actively working on delivering results. This feature allows for real-time responses, especially for lengthy answers. The Astra Assistants now supports this functionality in both managed service and open source codebase. Streaming runs provide a more user-friendly approach compared to the previous polling-based method. With streaming support, users can create more engaging and effective retrieval-augmented generation applications.

DataStax is integrating NVIDIA's NIM and NeMo Retriever microservices into its Astra DB, aiming to deliver high-performance retrieval-augmented generation (RAG) solutions with fast embeddings. This integration will help reduce latency in accessing structured and unstructured data for enterprise users of generative AI applications. RAG combines pre-trained language models with a retrieval system, enabling enterprises to leverage their own data while reducing hallucinations and improving specificity. DataStax is working with NVIDIA to solve the challenge of vectorizing existing and newly added unstructured data for large language model inference. The integration of NVIDIA's microservices with Astra DB provides fast embedding+indexing latencies, high operations per second, and lower operational costs. This collaboration aims to provide a fast vector DB RAG solution built on a scalable NoSQL database that can run on any storage medium.

esynergy developed Sales Copilot, an AI-powered conversational assistant for sales teams using Langchain and DataStax Astra DB. The application ingests customer data from SharePoint, encodes it into vectors, indexes in Astra DB for low-latency search, retrieves relevant chunks for a query, and passes them to Claude2 to generate responses. This retrieval-augmented setup allows generating contextual responses by conditioning on relevant data. The application seamlessly integrates with an organization's SharePoint instance, allowing it to ingest customer profiles, communication records, and case studies. Langchain plays a crucial role in the preprocessing stage, enabling efficient splitting of files into manageable chunks for processing. AWS Bedrock suite is used for text embedding models, transforming chunks into dense vector representations. The retrieval-augmented architecture ensures relevant data points are retrieved from Astra DB and used to generate informed responses. Using Langchain accelerated the development of a production-grade conversational assistant, offering benefits such as minimizing repetitive tasks and fostering more meaningful conversations with customers.

DataStax Astra DB now offers multi-region vector data support, which enables businesses to put relevant data in the right place for maximum responsiveness while delivering high availability. This feature reduces latency and facilitates seamless staging to production replication, improving user experiences, engagement, and success of GenAI initiatives. Multi-region support also ensures business continuity by maintaining application availability uptime and enabling rigorous testing in geographically diverse settings.

London-based Artera aims to transform how art is experienced on mobile platforms using cutting-edge generative AI technology. Its Art Discovery Platform enables users to load their artwork, curate personal collections, and share them within the platform. By integrating vector search, streaming, and other technologies, the platform allows effortless discovery of similar artworks or exploration of recommended pieces. Artera leverages DataStax Astra DB for its real-time data interactions and precise art-matching capabilities, enriching users' experience. The platform also offers audio comments and text search features to deepen the connection between art and its audience. Artera uses OpenAI's CLIP model for image embeddings and vector search capabilities, enabling personalized and intuitive art discovery experiences. Future updates will include Image Capture Pose Estimation and a Patronage System to support artists directly.

OpenAI CEO Sam Altman unveiled the OpenAI Assistants API beta preview on November 8th, prompting DataStax to develop an API-compatible service powered by Astra DB and its vector search capabilities. The service was launched eight days later and is now being open-sourced for users who want to see how it works or host it themselves. The assistant-api-server repo is a Python server app that relies on FastAPI, Pydantic, and the DataStax Python CQL driver. It uses LiteLLM for third-party LLM support and is mostly stateless with all authentication tokens and LLM provider configuration passed as http headers. The service has been running in production since November 15th, 2023, and users are encouraged to try the service, check out the code, and provide feedback on GitHub.

A new partnership between DataStax and Airbyte offers a DataStax Astra DB data destination in the Airbyte Cloud data pipeline, enabling developers to easily move data and generate embeddings within Astra DB. This accelerates the development of generative AI applications by shortening development cycles, simplifying data loading, and accelerating vector creation. The integration allows for easy ingestion of data into Astra DB, which can be prepared and kept up-to-date asynchronously from the overall application.

DataStax has achieved AWS Generative AI Competency with its Astra DB vector database, which provides all the necessary data to build generative AI applications. This new program from Amazon Web Services validates partners' expertise in helping organizations unlock the power of generative AI for creating innovative and competitive applications. The use of GenAI is becoming increasingly important for businesses seeking to stay ahead in today's fast-paced, highly competitive industry. DataStax Astra DB seamlessly integrates with AWS ecosystem, enabling developers to build and deploy production-ready GenAI applications quickly.