October 2024 Summaries

The article discusses leveraging data not incorporated into large language models (LLMs) by providing supplemental context when querying the data. It introduces Glean, a unified framework for indexing and searching company data, which significantly increases the potential power of retrieval-augmented generation (RAG) frameworks. The article then demonstrates how to set up a flow using the Langflow framework, which pulls data from the Glean Search API, and enables the construction of an easy-to-use RAG pipeline. It also highlights the importance of ensuring that any data accessible is safe to provide to LLM providers when using online models. The author concludes by encouraging readers to try the Glean Langflow Component for themselves and explore more ideas for modern AI applications.

GitHub Copilot has released a significant update to its Astra DB extension, enhancing database management capabilities, integrating Langflow support, and streamlining generative AI application development within the IDE. The updated extension allows developers to create new Astra DB databases directly through GitHub Copilot Chat and manage Langflow projects without leaving VS Code. This integration aims to improve productivity, code quality, and collaboration among developers of all backgrounds by providing an integrated, AI-assisted development environment.

Skypoint, a healthcare industry data analytics provider, has developed an innovative solution to transform vast amounts of unstructured data into actionable intelligence by leveraging vertical AI agents and industrial-scale AI. The company's journey in healthcare has led to the creation of a production-level system that bridges the gap between unstructured data and actionable insights. Skypoint uses DataStax's Astra DB, a fully managed vector database built on Apache Cassandra, to handle large amounts of unstructured data generated by healthcare systems and other regulated industries. The hybrid approach of AI agents and human oversight ensures the highest standards of data integrity while providing predictive analytics, sentiment scoring, and real-time dashboards for various stakeholders in the senior living market intelligence sector. Skypoint's vertical AI agents are revolutionizing the market by offering unmatched cost-efficiency and scalability across industries that have been underserved by technology.

Streaming in AI applications can enhance their responsiveness and interactivity, but implementing streaming presents challenges. LangChain simplifies the process of adding streaming support to GenAI applications by providing out-of-the-box components and a simple API for handling requests/responses, validation, error checking, scaling, parallelization, and other operational tasks. LangChain also supports streaming LLM responses, making it easier to ship scalable apps with high volume simultaneous LLM transactions. However, developers must still consider issues like performance optimization and managing latency when using streaming LangChain. Langflow, a visual tool for LangChain, further simplifies the process of adding streaming support by providing a user-friendly interface for building GenAI applications. Overall, integrating streaming with AI applications can improve user satisfaction and adoption while reducing development time and code complexity.

Knowledge graphs are being utilized to enhance the results of retrieval-augmented generation (RAG) applications, with most examples demonstrating how to build a knowledge graph from a small number of documents. The typical approach involves extracting fine-grained, entity-centric information, which does not scale well due to time and cost constraints when dealing with large datasets. Content-centric knowledge graphs, such as GraphVectorStore, offer an easier and more efficient alternative by allowing links between chunks. This article presents a comparison of the two approaches using a subset of Wikipedia articles from the 2wikimultihop dataset. The content-centric approach is shown to be significantly faster and less expensive than the entity-centric method when loading large datasets, with parallelism further reducing processing time. Additionally, the content-centric approach produces more accurate and relevant answers to questions posed over the loaded data. Overall, GraphVectorStore offers a practical solution for building knowledge graphs at scale for RAG applications.

OpenOrigins is developing a platform to help archivists quickly and efficiently find relevant videos in digital media archives by providing advanced search capabilities. The company is considering two technological approaches: frame-by-frame analysis of videos using image embeddings, and multimodal embeddings. While the former offers high accuracy in multimodal semantic search but may miss temporal context or changes between frames, the latter leverages Google's multimodal embedding model to enable users to search videos using images, text, or videos, converting all inputs into a common embedding space. This approach efficiently manages large datasets with temporal context and supports multiple input types for search queries, making it an excellent choice for complex search scenarios.

Artificial Intelligence (AI) has revolutionized many industries, with retail and e-commerce being among the most impacted sectors. The evolution of AI in retail has moved from rule-based systems to more sophisticated AI-driven solutions that leverage machine learning for personalization and real-time recommendations. AI adoption is still in its early stages, but tools like DataStax Langflow are simplifying the process of building AI applications for developers. Use cases such as personalization, automation, optimization, and improved customer experiences are driving the adoption of AI in retail. As AI continues to evolve, it will likely become even more central to how businesses operate, improving efficiency and enhancing overall customer satisfaction.

The relationship between Apache Cassandra and DataStax drives innovation and benefits the entire community. With hundreds of engineers working on Cassandra across various employers, including notable concentrations at DataStax, Apple, Netflix, and Netapp, a structured approach called the Cassandra Enhancement Proposal (CEP) process ensures large contributions are thoroughly vetted before inclusion in the project. This careful balance between rapid innovation and long-term stability has led to significant contributions from DataStax, such as Storage Attached Indexes (SAI), trie indexes, unified compaction strategy, and vector search. By staying closely aligned with the upstream version of Cassandra, companies like DataStax avoid compatibility issues and maintenance burdens while contributing their changes back to the main project. This collaboration fosters a close relationship between commercial entities and the broader Cassandra community, driving innovation and ensuring stability for users.

Retrieval-augmented generation (RAG) is an AI technique that enhances output from large language models by providing real-time context when generating responses. Graph RAG takes this concept further by organizing information into interconnected webs, allowing for richer and more nuanced connections between data points. While traditional vector search methods can struggle with capturing all important relationships within unstructured data, graph RAG offers a solution by leveraging knowledge graphs to model these relationships explicitly. However, implementing full knowledge graphs comes with significant challenges such as modeling complexity, unstructured data challenges, and maintenance overhead. Graph RAG provides an accessible alternative that augments existing vector information with links, offering many of the benefits of knowledge graphs without the same level of implementation complexity. This technique has shown promise in various domains and can be implemented using LangChain-based graph RAG, which requires minimal code changes for a significant boost in retrieved information accuracy.

In this blog post, the author demonstrates how to extend notion-clone, an open-source clone of Notion, to use vector search for retrieval-augmented generation (RAG) with Mongoose and Astra DB. RAG enables the LLM to answer questions based on notes entered into notion-clone by finding relevant content using vector search. The implementation includes changes to the backend, such as implementing a $vector property using Mongoose save() middleware and converting HTML to plain text for accurate embeddings. Additionally, rate limiting support is added to prevent abuse. Overall, this combination of Astra DB and Mongoose provides an efficient vector search integration within a sophisticated database framework.