April 2024 Summaries

Milvus 2.4 introduces multi-vector search and hybrid search capabilities, allowing simultaneous queries across multiple vector fields and integrating the results with re-ranking strategies. Hybrid search is a process of conducting searches across various vector fields within the same dataset. This tutorial demonstrates how to leverage Milvus's hybrid search capabilities using the eSci dataset and BGE-M3 model. The steps include preparing the dataset, generating embeddings with BGE-M3, setting up a Milvus collection, inserting data into the collection, and executing hybrid searches in Milvus.

Implementing Retrieval Augmented Generation (RAG) technology in chatbots can significantly enhance customer support by combining large language models with knowledge stored in vector databases from various fields. RAG systems consist of two core components: the Retriever and the Generator, which work synergistically to handle complex queries effectively. Compared to traditional LLMs, RAG offers several advantages such as reduced hallucination issues, enhanced data privacy and security, and real-time information retrieval. While advancements in LLMs also address these challenges, RAG remains a robust, reliable, and cost-effective solution due to its transparency, operability, and private data management capabilities. RAG technology is often integrated with vector databases, leading to the development of popular solutions like the CVP stack. Vector databases are favored in RAG implementations for their efficient similarity retrieval capabilities, superior handling of diverse data types, and cost-effectiveness. Ongoing engineering optimizations aim to enhance the retrieval quality of vector databases by improving precision, response speed, multimodal data handling, and interpretability. As demand for RAG applications grows across various industries, RAG technology will continue to evolve and revolutionize information retrieval and knowledge acquisition processes.

This guide provides an overview of open-source licensing in the context of AI technology, specifically vector databases and large language models (LLMs). Open source allows creators to make software or hardware available for free, often developed and maintained by community efforts. Understanding different license types is crucial as changes can significantly impact companies and businesses that rely on open-source software. The benefits of open-source vector databases and LLMs include rapid prototyping, increased trust and transparency, and reduced costs for developers. Various types of licenses exist, including permissive licenses (e.g., MIT License), copyleft licenses (e.g., GNU General Public License), weak copyleft licenses (e.g., GNU Affero General Public License), non-commercial licenses (e.g., Creative Commons Non-Commercial License), and public domain releases. Key organizations like the Open Source Initiative, Free Software Foundation, and Apache Software Foundation govern open-source licensing standards. The degrees of openness in different licensing models influence collaboration, innovation, and transparency in AI development. Licensing plays a vital role in shaping AI technologies' trajectory by governing accessibility, adaptability, and equitable distribution.

LangChain and Zilliz Cloud offer an effective combination to create AI-powered search systems. These systems use natural language processing (NLP) and machine learning algorithms to enhance the accuracy and relevance of information retrieval across business-specific data. With the rise of generative models, AI-powered search applications have become more prominent compared to traditional search methods. However, ensuring user privacy in these applications is critical due to ethical and legal implications. The integration of LangChain with Zilliz Cloud allows for the creation of custom search engines that prioritize data privacy while offering tailored solutions based on specific needs and data. Both tools provide robust frameworks for ensuring privacy and safety when utilizing large language models (LLMs), effectively preventing private data misuse and generating harmful or unethical content.

Vector search is a technique used in data retrieval for RAG applications and information retrieval systems to find items or data points that are similar or closely related to a given query vector. While many vector database providers market their capabilities as easy, user-friendly, and simple, building a scalable real-world application requires considering various factors beyond the coding, including search quality, scalability, availability, multi-tenancy, cost, security, and more. To effectively deploy your vector database in your RAG application production environment with Milvus, follow these best practices: design an effective schema, plan for scalability, and select the optimal index and fine-tune performance.

Milvus is an open source vector database that enables efficient search over large amounts of data. When deploying Milvus, it's crucial to select the optimal configuration to ensure efficient performance and resource utilization. Key points to consider include index selection, balancing memory usage, disk space, cost, speed, and accuracy; segment size and deployment configuration; and additional customization options available in the Enterprise version of Zilliz Cloud. The Milvus sizing tool provides a starting point for these configurations, but users should also consider their specific needs and requirements when choosing an index algorithm or segment size.

This guide explores Milvus, an open-source vector database known for its horizontal scalability and fast performance. At the core of Milvus lies its robust storage system, which comprises meta storage, log broker, and object storage. The architecture is organized into four key layers: access layer, coordinator service, worker nodes, and storage. Milvus uses three main storage components to ensure data integrity and availability: meta storage (etcd), object storage (MinIO), and a log broker (Pulsar or Kafka). To evaluate and optimize the performance of Milvus storage, it is crucial to monitor disk write latency, I/O throughput, and disk drive performance. The guide provides recommendations for selecting appropriate block storage options from various cloud providers and offers strategies to enhance MinIO's throughput performance by using SSD or NVMe-type drives.

This tutorial demonstrates how to build retrieval augmented generation (RAG) applications using large language models (LLMs) without relying on OpenAI. The process involves serving embeddings with BentoML, inserting data into a vector database for RAG, setting up an LLM for RAG, and providing instructions to the LLM. Key components include BentoML for serving embeddings, OctoAI for accessing open-source models, and Milvus as the vector database. The example uses BentoML's Sentence Transformers Embeddings repository, a local Milvus instance using Docker Compose, and the Nous Hermes fine-tuned Mixtral model from OctoAI for RAG.

Milvus is an open-source vector database designed to efficiently manage and retrieve high-dimensional vector data, making it ideal for use in artificial intelligence and machine learning applications. By integrating Milvus with Spring AI, developers can leverage advanced search capabilities and optimize their applications' performance and scalability. This integration allows users to perform complex queries and similarity searches quickly and accurately, enhancing user experiences and enabling more intelligent application behavior. Key features of Milvus include support for various indexing strategies, compatibility with different metric types, and the ability to handle large volumes of vector data through partitioning and sharding. Use cases for Milvus in Spring AI applications span across recommendation systems, content search engines, image and video recognition, and AI-driven chatbots and customer support.

This guide provides a beginner's approach to setting up a Retrieval Augmented Generation (RAG) system using Ollama, Llama 3, Milvus, and Langchain. The RAG technique enhances large language models (LLMs) by integrating additional data sources. In this tutorial, we will build a question-answering chatbot that can answer questions about specific information. Key components of the setup include indexing data using Milvus, retrieval and generation with Llama 3, and interaction with data using Langchain. The guide assumes familiarity with Docker and Docker Compose, as well as installation of Milvus Standalone, Ollama, and other necessary tools.

Milvus is an open-source vector database with significant traction in Generative AI and RAG use cases. It offers flexible deployment options, including local and cloud (Zilliz) services. The main dependencies for Milvus standalone server include FAISS, etcd, Pulsar/Kafka, Tantivy, RocksDB, Minio/S3/GCS/Azure Blob Storage, Kubernetes, StorageClass, Persistent Volumes, Prometheus, and Grafana. The Docker image size for Milvus standalone container is around 300MB. It has a frequent release cycle with approximately one major release per month. Six SDKs are available in Python, Node, Go, C#, Java, and Ruby. Understanding these details can help organizations better plan and prepare for integrating Milvus into their technology stack.

The future of vector databases is closely tied to the evolution of product requirements and user demands. Key areas of development include cost-efficiency, hardware advancements, collaboration with advanced machine learning models, prioritizing retrieval accuracy, optimizing for offline use cases, expanding feature sets for diverse industries, and more. As AI continues to mature, these advancements will enable vector databases to support a broader range of applications across various sectors, enhancing their overall functionality and versatility in production environments.

Streamlining data processing using Zilliz Cloud Pipelines involves examining document chunking, a component of transforming unstructured data into a searchable vector collection. The platform enables use cases with semantic search in text documents and provides a critical building block for Retrieval-Augmented Generation (RAG) applications. Zilliz Cloud Pipelines include various functions like SEARCH_DOC_CHUNK, which convert the query text into vector embedding. It will then retrieve the top-K relevant document chunks, making it easier to find the related information based on the query’s meaning. The engineers at Zilliz designed Zilliz Cloud Pipelines to transform unstructured data from various sources into a searchable vector collection for busy Gen AI developers. This pipeline will take unstructured data, split it, convert it to embeddings, index it, and store it in Zilliz Cloud with the designated metadata.

Charles Xie, CEO of Zilliz, discusses the evolution and future of AI and its influence on vector databases. He highlights how Zilliz developed Milvus, a vector database, before the advent of large language models (LLMs), emphasizing the importance of data management for unstructured data. The article also explores the transition from enterprise-centric to democratized AI, as well as the significance of vector databases in the age of Foundation Models and LLMs. Furthermore, it delves into the role of Milvus 3.0 in enhancing retrieval accuracy for RAG systems and how ChatGPT and vector databases complement each other in semantic search. Lastly, Xie shares his vision for Affordable General Intelligence within five years, aiming to make AI-solutions accessible to all individuals and businesses.

This blog discusses the use of Retrieval Augmented Generation (RAG) applications with open-source tools such as Ray Data and Milvus. The author highlights the performance boost achieved using Ray Data during the embedding step, where data is transformed into vectors. By using just four workers on a Mac M2 laptop with 16GB RAM, Ray Data was found to be 60 times faster than Pandas. The blog also presents an open-source RAG stack that includes BGM-M3 embedding model, Ray Data for fast, distributed embedding inference, and Milvus or Zilliz Cloud vector database. The author provides a step-by-step guide on how to set up these tools and use them to generate embeddings from data downloaded from Kaggle IMDB poster. Additionally, the blog discusses the benefits of using bulk import features in Milvus and Zilliz Cloud for efficient batch loading of vector data into a vector database.

This guide provides step-by-step instructions on setting up Grafana and Loki to effectively monitor Milvus deployments. Milvus is a distributed vector database designed for storing, indexing, and managing massive embedding vectors. Grafana is an open-source platform for monitoring and observability, while Loki pairs with Grafana as a log aggregation system. Together, they offer a solid monitoring setup for Milvus and beyond. The prerequisites include Docker, Kubernetes, Helm, and kubectl. After setting up the K8s cluster, users can deploy Grafana and Loki using Helm. Finally, configure Grafana data sources and dashboard to visualize and query logs effectively.

The cost of open source vector databases can be complex and challenging to quantify. Engineers often start projects using free software like Milvus, but hardware costs soon arise. Running a distributed database requires setting up dependencies such as Kafka or Pulsar for WAL, etcd for metadata storage, and Kubernetes for orchestration. Additionally, costs include load balancers, monitoring and logging tools, EC2 instances for worker nodes, and storage solutions like S3 or Azure Blob. Some aspects of running an open-source vector database are difficult to quantify, such as capacity planning, setup phase tasks, routine maintenance, troubleshooting latency issues, and disaster recovery plans. Other costs include time to market, engineering morale and retention, and risk mitigation. To assess costs in vector database management, performance tests should be conducted to gather data on how the database handles real-life workloads. Optimizing for cost involves adopting dynamic scaling, adjusting recall accuracy, latency, and throughput according to project needs, and using MMap to store less data in memory. The decision on how to manage a vector database ultimately depends on comparing costs and making an intelligent economic choice based on the most cost-effective option.

Redis, an open-source database software, has transitioned from the BSD license to the Server Side Public License (SSPLv1), causing some controversy. This change may lead to multiple Linux distributors dropping Redis from their codebases, but alternative options like Valkey and Microsoft's Garnet are available. The shift in open-source licensing has been driven by cloud computing's impact on the traditional business model of open-source software companies. Some open-source projects have adopted more restrictive licenses to protect their profits, while others continue to offer permissive licenses and focus on commercial services. Companies like Zilliz are finding new ways to balance open-source and commercialization by offering unique capabilities in their managed services while maintaining compatibility with the open-source API.

Charles, CEO of Zilliz, discusses the evolution and future of vector databases in AI applications. He explains that vector databases are designed to manage and query unstructured data like images, videos, and natural languages through deep learning algorithms and semantic queries. They are widely used in recommendation systems, chatbots, and semantic search. The current landscape of vector databases includes purpose-built ones like Milvus, traditional databases with a vector search plugin like Elasticsearch, lightweight vector databases like Chroma, and more technologies with vector search capabilities like FAISS. Charles shares insights into building the Milvus vector database system, emphasizing its support for heterogeneous computing, both vertical and horizontal scalability, and offering a smooth developer experience from prototyping to production. He also provides guidance on choosing the right vector database for businesses based on performance requirements and projected data volume growth. Charles predicts that future vector databases will extend their capabilities beyond similarity-based search to include exact search or matching, as well as support additional vector computing workloads like clustering and classification.

A group of four strangers, including a Product Manager, full-stack developers, and an AI enthusiast, came together during a monthly Hackathon in Seattle to build the SaveHaven project. The team developed a Retrieval Augmented Generation (RAG) app called SaveHaven that helps individuals contest property and income tax assessments by leveraging technologies like LlamaIndex, Milvus, and GPT from OpenAI. By automating data collection and analysis from public records, the app simplifies the tax appeal process for the general public. The team's experience serves as an example for future entrepreneurs to build meaningful innovations using GenAI technologies.

The author discovered their love for open-source image-generating AI systems and started searching through webpages to find cool images and the prompts that made them. They used those prompts to make their own images, but it took a lot of time. To speed up the process, they downloaded millions of prompts and put them into a Milvus vector database. The system was able to fetch similar results based on simple prompts entered into a UI. Users found that the system produced better results than what they were doing before with their regular prompts. The author chose Milvus for performance reasons, as it was five times faster than pgvector with almost the same code. They also added instructions telling the LLM that it was a prompt engineer and provided some example conversation history to get it to start producing wonderful images. The next step is to add the same function for negative prompts, which have a positive influence on how prompts can be used to generate images.