May 2024 Summaries

Dr. Damon Abraham, a behavioral scientist with a PhD in Psychology, has researched how reappraising an image can shift our valence and arousal emotions. His project aims to create a stimuli repository for experimental psychology using LLM and vector database technologies. The research involves collaboration with the University of Denver and other institutions, and it seeks to develop an open-source normative database of images and techniques to measure the dynamic psychological distance between images and their potential for successful reappraisal. In his presentation at the Zilliz Unstructured Data Meetup in Seattle on February 13, 2024, Dr. Abraham discussed how different contextual interpretations of images can change our feelings, providing insights into emotional regulation. The study also explores the concept of 'reappraisal affordances,' which examines how an image's inherent semantic properties and range of associations may affect its capacity for reinterpretation.

Tim Spann, an advocate of Open Source projects, has been working on the intersection of streaming and AI at Zilliz. He emphasizes the importance of a database for AI that can store and query any type of data in any mode needed. With Milvus, Towhee, Attu, and integrations with Kafka and LlamaX frameworks, Spann aims to build up a global group of unstructured data engineers and data superstars. He believes the future will see a rise in unstructured data engineering and processing like Spark, Flink, and Kafka for structured and semistructured data. The need for powerful, fast ways to do unstructured data processing and Vector ETL is evident and growing.

This tutorial demonstrates how to build an AI Agent using LangChain, Milvus, and OpenAI. The agent performs Retrieval Augmented Generation (RAG) tasks by retrieving information from a vector database like Milvus. Additionally, the monitoring tool Portkey is used to track token usage, token count, and request latency. The tech stack includes LangChain for orchestration, Milvus as a vector database, Portkey for monitoring, and OpenAI for the Language Learning Model (LLM).

Retrieval Augmented Generation (RAG) is an approach in Generative AI that utilizes data to enhance the knowledge of Language Learning Model (LLM) generators, such as ChatGPT. RAG consists of two LLMs: embedding and generator models, both used in inference mode. The HuggingFace MTEB leaderboard provides a comprehensive list of text embedding models, where users can filter by language or specialty domain like law. Users should be cautious when selecting models as some may be overfitted, resulting in deceptively high rankings. ResNet50 is a popular Convolutional Neural Network (CNN) model for image data and PANNs are commonly used embedding models for audio data. Multimodal embedding models like SigLIP or Unum can handle text, image, audio, or video data simultaneously. For multimodal applications involving sound or video, a generative LLM is often employed to convert the input into text before using RAG techniques.

In February 2024, Yury Malkov discussed Approximate Nearest Neighbor (ANN) and its key role in recommender systems at the SF Unstructured Data Meetup. ANN search is already integrated into the production stacks of popular tools worldwide. The talk covered the key concepts and background that have driven ANN's adoption in large-scale recommender systems. Yuri Malkov, a genius physicist, laser researcher, and inventor of HNSW (a graph-based indexing algorithm), now works as a Research Scientist for OpenAI. ANN search algorithms use various indexing techniques to return approximate nearest neighbors, making them core to many applications and technologies that are customer facing today. From search engines like Google to social media sites, ANN and recommender systems are already integrated through the stack in production. Yuri notes that many mature ANN solutions exist, including LSH, graph-based indexes like HNSW and SCANN, quantization-based indices like IVF_PQ and IVF_HSNW, DiskANN, and ANNOY. ANN benchmarking is done through platforms like ANNBenchmarks, which evaluate various approximate nearest neighbor search algorithms and provide results split by distance measure and dataset on their website. The performance metrics include recall rate and queries per second (QPS). More QPS indicates better performance. The Milvus team built Knowhere, an open-source vector execution engine that incorporates several vector similarity search libraries like Faiss, Hnswlib, and Annoy. It controls on which hardware (CPU or GPU) to execute index building and search requests. Cardinal is Zilliz's core vector search engine, offering a threefold increase in performance compared to the previous version. Recommender systems have large market potential due to their ability to generate consumer behavior. Typical challenges at scale include generability, handling huge corpuses, and efficiency. Traditional recommender systems use a multi-staged funnel approach with candidate generation, lightweight ranking, and full ranking stages. Novel solutions for item-query incompatibility include L2 distance on data vectors, bipartite graph ranking, text-focused graph re-ranking, and cascaded graph search. ANN algorithms have seen extensive implementation due to their good enough matching, flexibility, and maturity. Further resources are available through Yury Malkov's talk on YouTube.

Milvus, an open-source vector database, enhances the development of Retrieval Augmented Generation (RAG) applications by streamlining processes and improving efficiency. Its integration with popular embedding models simplifies text transformation into searchable vectors, while its hybrid search capability supports multimodal data retrieval. Additionally, Milvus offers a cost-effective solution for managing large knowledge bases through minimizing memory consumption, implementing tiered data storage, and leveraging intelligent caching and data-sharding techniques. Overall, Milvus helps developers build faster, more accurate, and cost-efficient RAG applications.

This tutorial demonstrates how to build a Multimodal Retrieval Augmented Generation (RAG) System, which allows the use of different types of data such as images, audio, videos, and text. The system utilizes OpenAI CLIP for understanding the connection between pictures and text, Milvus Standalone for efficient management of large-scale embeddings, Ollama for Llama3 usage on a laptop, and LlamaIndex as the Query Engine in combination with Milvus as the Vector Store. The tutorial provides code examples available on Github and explains how to run queries that can involve both text and images.

Zilliz has achieved AWS Generative AI Competency Partner designation, demonstrating its commitment to advancing generative AI technologies. As an AWS Differentiated Partner, Zilliz provides critical infrastructure and best practices for implementing transformative generative AI applications such as image retrieval, video analysis, NLP, recommendation engines, customized search, intelligent customer service, fraud detection, and more. The company's expertise combined with the scalability, performance, and security of AWS Cloud enables customers to unlock new possibilities and gain a competitive edge in their industries. Zilliz remains dedicated to fostering collaboration, knowledge sharing, and responsible AI practices while working closely with AWS and its customers to shape the future of generative AI.

This blog post discusses various ways to run the G-Generation part of Retrieval Augmented Generation (RAG) using different models and inference endpoints. The author provides step-by-step instructions on how to use Llama 3 from Meta, Mixtral from Mistral, and the newly announced GPT-4o from OpenAI. They also cover running these models locally or through Anyscale, OctoAI, and Groq endpoints. Additionally, the author explains how to evaluate answers using Ragas and provides a summary table of results for each model endpoint. The conclusion emphasizes the importance of considering answer quality, latencies, and costs when choosing an appropriate model and inference endpoint for the G-Generation part of RAG.

Milvus, an open-source vector database designed to store, index, and search massive amounts of vector data in real-time, can be integrated with LLMs in a generative feedback loop. This allows for continuous learning and improvement of the AI system. Feedback loops are crucial in ensuring the ongoing refinement of model outputs in AI systems, offering benefits such as adaptability to new data, reduced bias and errors, personalized model outputs, and enhanced creativity and innovation. Milvus's features make it suitable for enhancing the data-handling capabilities of LLMs, particularly in scenarios where feedback loops are used to refine predictive and generative accuracies.

DSPy is a programmatic framework designed to optimize prompts and weights in language models (LMs), particularly in use cases where you integrate LMs across multiple pipeline stages. It provides various composable and declarative modules for instructing LMs in Pythonic syntax. Unlike traditional prompting engineering techniques that rely on manually crafting and tweaking prompts, DSPy learns query-answer examples and imitates this learning to generate optimized prompts for more tailored results. This allows for the dynamic reassembly of the entire pipeline, explicitly tailored to the nuances of your task, thus eliminating the need for ongoing manual prompt adjustments. DSPy has been integrated into the DSPy workflow as a retrieval module in the form of the MilvusRM Client, making it easier to implement a fast and efficient RAG pipeline. In this demonstration, we'll build a simple RAG application using GPT-3.5 (gpt-3.5-turbo) for answer generation. We use Milvus as the vector store through MilvusRM and DSPy to configure and optimize the RAG pipeline.

This blog discusses resource allocation for Milvus, an open-source vector database. It provides reference architectures based on specific numbers of users or requests per second (RPS) and different mixes of READ and WRITE operations. The article emphasizes the importance of understanding workload characteristics to determine Milvus' computational power and memory requirements. It also outlines a method for estimating resource needs, load testing, benchmarking, and concludes with recommendations for resource allocation based on data size and QPS requirements.

Zilliz and Azure OpenAI have integrated to redefine similarity and semantic search, offering remarkable speed, intelligence, and safeguards. The collaboration combines Azure OpenAI's advanced generative AI capabilities with Zilliz's scalable search solutions, enhancing AI search functionalities and data retrieval. This partnership enables seamless integration of AI models and scalable search solutions for developers. Zilliz is a specialized data management system optimized for managing high-dimensional vector data on a large scale, while Azure OpenAI provides additional features like private networking, regional availability, and responsible AI content filtering. The integration of these technologies offers robust data storage, sophisticated indexing options, and comprehensive similarity metrics and retrieval mechanisms, enabling developers to create scalable and efficient AI-driven search solutions.