December 2024 Summaries

In 2024, Apache Cassandra experienced significant growth and evolution, with over 90,000 new lines of code added by the community and more than 70 active contributors driving progress. The introduction of Storage Attached Indexes (SAI) in Cassandra 5.0 improved query flexibility and enabled complex applications to be built on top of the database. Vector search capabilities made their debut in Cassandra 5, positioning the project perfectly for AI and ML applications. A unified compaction strategy was also introduced, adapting automatically to workload demands and providing intelligent optimization. The project's security features, including dynamic data masking, were further enhanced. User surveys showed that 85% of users are either in the cloud or moving towards it, indicating Cassandra's architectural decisions were correct from the start. The growth of AI use cases is projected to be 350% next year, with over 50% of respondents already running generative AI in production. The enthusiasm for Cassandra 5.0 was evident, with 62% of users planning to upgrade within the next year. Overall, the momentum for Cassandra continues to build, with the database adapting to new workloads while staying true to its roots of scalability and reliability.

A vector database is used to help add context to generative AI application prompts, and can be used with LangChain, a popular GenAI framework, to build more accurate applications in less time. Using a local vector database can save money and simplify setting up a local dev stack, but requires careful management to avoid expensive cost overruns. LangChain supports composing calls to large language models (LLMs) with other AI app components using a simple programming syntax, and offers multiple components for retrieval-augmented generation (RAG). A vector database converts data into mathematical vector embeddings, allowing for searching approximate matches in a multi-dimensional vector space. Developers can use Docker containers to spin up local vector database instances, or opt for a cloud-hosted service like Amazon Web Services or Astra DB. Local development has challenges, including the need for a transition plan and potentially requiring too many resources to load and run large datasets. An alternative to local development is using a serverless vector database like Astra DB, which provides an affordable option with seamless integration with LangChain via the Astra DB connector.

We created a multiplayer movie trivia game that combines real-time gaming with AI-generated questions using RAG (retrieval-augmented generation) system, Langflow to generate movie-related questions, and PartyKit for rapid iteration. The game uses a client-server architecture with a central server maintaining the true game state, ensuring automatic load balancing at production scale with Cloudflare Durable Objects. We experimented with multiple LLMs and providers, landing on MistralAI's open-mixtral-8x7b model to generate believable fake movie quotes, injecting suspense and uncertainty among players. The game features real-time multiplayer infrastructure using PartyKit, allowing for instant setup and iteration. Building this game taught us lessons about Astra DB, Langflow, and PartyKit, with valuable feedback gathered that has already been actioned.

Vector stores and LangChain are technologies that can increase response accuracy and speed up release times when used together in GenAI apps. A typical GenAI app consists of multiple components, including large language models, response parsers, verifiers, external data stores, cached data, agents, and integrations with third-party APIs. LangChain is a framework that represents all components as objects and provides a simple language to assemble them into a request/response processing pipeline. Retrieval-augmented generation (RAG) takes a user's query and gathers additional context from external data stores to improve LLM responses. Vector databases excel at storing high-dimensional data with retrieval via semantic search, allowing for low-latency queries and timely, accurate domain-specific responses. When choosing a vector store, factors such as ease of use, performance, accuracy, relevancy, and system reliability should be considered. A serverless vector store can address reliability concerns by scaling automatically to meet demand. DataStax offers solutions that enable GenAI app developers to add RAG functionality with minimal effort, including Apache Cassandra and Astra DB, a zero-friction drop-in replacement for Cassandra made available as a fully managed service.

Vector databases are critical components of generative AI applications and offer high-dimensional data storage and support for queries across vector data. They convert data into numerical embeddings, employ vector search algorithms, and return a larger volume of data with a broader scope than traditional keyword searches. Vector databases are well-suited to use cases such as building recommendation engines and retrieval-augmented generation (RAG). However, they also have the downside of being yet another element of your architecture that can become a point of failure. AWS provides various options for hosting vector databases, including managed services like Amazon RDS for PostgreSQL and Aurora, serverless services like Amazon Aurora Serverless, and self-hosted options such as Apache Cassandra and DataStax's Astra DB. Each option has its pros and cons, and the choice depends on staffing levels, support availability, and budget. Managed services simplify deployment and maintenance but require someone to monitor and fine-tune them for performance, while serverless services offer zero infrastructure management but are generally more expensive. Self-hosted options provide full control but require significant expertise and resources. AWS offers multiple options for hosting vector databases, allowing developers to choose the best fit for their use case and requirements.

Agents are smart systems designed to get things done on their own, powered by large language models (LLMs), and can analyze a situation and figure out the best next move all on their own. They decide what to do next based on the situation and the tools they have access to, don't wait for you to tell them every step, and are decision-makers with reasoning capabilities. An agent acts like a smart assistant that uses different tools to get tasks done, such as pulling in real-time stock prices or scheduling meetings. They use a language model to decide what actions to take and in what order, and can even use the output from one tool to make the input for another better. Agents are useful for multitasking wizards that save time by connecting tools and systems you use daily, and can handle complex workflows with multiple specialized agents working together. Langflow is an open-source framework that makes creating multi-agent systems and retrieval-augmented generation (RAG) setups straightforward and fun, with a user-friendly interface that allows developers to drag and drop AI components like building blocks to create something that works. Agents in Langflow can be customized with tools such as calculators, web-searching superpowers, and personalized advice, and can be monitored with tools like LangSmith and LangWatch for valuable insights into their performance.

Langflow is an open-source tool offered by DataStax, allowing developers to build PDF chatbots with a low-code approach. It provides a visual workflow creator where users can assemble components without writing code, and supports various features such as API keys, text extraction settings, and vector search capabilities. The platform enables users to integrate their chatbot into frontend user interfaces via an HTTP API, and also allows for storing and retrieving document content using Astra DB's vector database.

We showcased the capabilities of the DataStax AI platform at AWS re:Invent, highlighting its ability to simplify and streamline building, deploying, and scaling generative AI applications on AWS. A powerful collaboration with Wikimedia Deutschland was unveiled, leveraging the platform to make Wikidata available as an embedded vectorized database, reducing development time from 30 days to just three days. Our cloud-native database-as-a-service, Astra DB, and drag-and-drop low-code development environment, Langflow, were also demonstrated in action, allowing visitors to build and deploy generative AI and retrieval-augmented generation apps with ease. Hands-on workshops and sessions on various topics were hosted, including deploying Apache Cassandra using Amazon Elastic Kubernetes Service and building RAG apps, and a demo of the IDE was shown for our developer relations team's AI-based trivia web app, Unreel. The event highlighted the partnership between DataStax and AWS, as well as other partners like NVIDIA, to empower developers with an AI platform that simplifies application development and deployment on AWS.

DataStax is celebrating the holiday season by giving away $1,000 per day to developers who create content around Astra DB and/or Langflow from December 12th to December 24th. To enter, participants must create human-created content featuring either or both technologies, post it on one of several specified platforms, and share it on social media with the appropriate tags. Winners will be selected through a random drawing each day, with an additional people's choice winner chosen at the end of the event based on likes. Entrants must be 18 years or older and reside in the U.S., excluding DataStax employees and their immediate family members.

Wikidata, a central repository for structured data used across Wikimedia projects, is partnering with DataStax to explore advanced methods for scaling semantic data search. This collaboration aims to expand the possibilities for working with large-scale, high-quality datasets and enhance data processing, storage, and usage. The partnership will also support the development of an equitable and validated approach to generative AI by leveraging Wikidata's knowledge graph. By offering a vector search API service that integrates with Wikidata's search API and UI, developers can build projects on top of Wikidata's knowledge graph, fostering a collaborative environment for the open-source machine learning community.

Datastax is dedicated to providing secure and streamlined database connectivity for its SaaS customers through integrations with AWS PrivateLink and VPC Lattice. The company's Astra DB, a hybrid vector database, seamlessly integrates with AWS PrivateLink to offer a private and secure networking experience. This integration ensures data security within the AWS ecosystem by eliminating the need for data to traverse the public internet. Datastax is committed to enhancing its private connectivity offerings through continuous integration of AWS PrivateLink and VPC Lattice into Astra DB, aiming to simplify setup processes, reduce operational costs, and provide efficient ways for customers to connect to their databases.