January 2024 Summaries

James Luan, VP of Engineering at Zilliz, discusses the company's commitment to innovation and community collaboration through open-source projects like Milvus. The four essential freedoms of open source, as emphasized by Richard Stallman, guide their approach to feature deployment from Milvus to Zilliz Cloud. They follow three fundamental principles: iteration with precision, testing the waters, and quality over speed. Despite occasional delays in feature deployment, they prioritize maintaining a robust and reliable platform while encouraging community feedback for continuous improvement.

In 2023, vector databases gained popularity due to the widespread adoption of ChatGPT and other large language models (LLMs). Zilliz Cloud, a vector database service, has seen increased usage in retrieval-augmented generation systems as well as various search and retrieval applications. The platform aims to help computers understand human-generated data such as text, images, bank transactions, and user behaviors. Zilliz Cloud recently introduced new features like range search, multi-tenancy & RBAC, up to 10x improved search & indexing performance, and more in response to customer demand. These enhancements have proven critical for users developing applications that require a purpose-built vector database supporting essential database features and various workloads. Three real-world use cases demonstrate the importance of these new features: efficient autonomous agents, product recommendation systems, and AI-powered drug discovery. In each case, Zilliz Cloud's performance optimizations, adaptability, and range search feature have enabled users to overcome challenges in their respective applications. The platform's ability to handle diverse data types and workloads makes it a valuable tool for developers working with vector databases.

Zilliz has introduced new features to its cloud product, enhancing vector search performance and ensuring enterprise-grade security. The latest updates include the Cardinal Search Engine, which delivers a 10x performance boost; Milvus 2.3, offering advanced vector search capabilities for production workloads; GCP Marketplace integration, simplifying budget planning, payment, and procurement processes; and the Databricks Connector, enabling data migration and transformation without custom code. Additionally, Zilliz Cloud now supports role-based access control (RBAC) across both control and data layers for enhanced security and compliance.

This blog delves into the concepts of sharding, partitioning, and segments in distributed databases like Milvus. Sharding refers to horizontal data partitioning across multiple servers, enabling faster writing by utilizing distributed systems. Partitioning organizes data for efficient retrieval, optimizing targeted reads. Automatic partitioning is recommended as it minimizes errors and ensures optimal performance. Each shard and partition has segments of data, with growing and sealed segments being the smallest unit in Milvus for load balancing. The default segment size is 512 MB, but adjustments should only be made if there are large machine resources available.

The BigANN challenge is an important competition in the vector search domain, fostering the development of indexing data structures and search algorithms. Zilliz's solution dominated all four tracks of BigANN 2023, achieving a remarkable up to 2.5x performance improvement. This year's BigANN introduced more significant challenges with larger datasets and complex scenarios across four tracks: filtered, out-of-distribution, sparse, and streaming variants of ANNS. Zilliz's solution is based on graph algorithms and optimizations driven by the specific characteristics of each track. The company plans to integrate these insights into their products, extending their impact on a broader range of issues.

This post discusses the creation of a Conversational Retriever Augmentation Generator (RAG) application without using OpenAI. The tech stack includes LangChain, Milvus, and Hugging Face for embedding models. The process involves setting up the conversation RAG stack, creating a conversation, asking questions, and testing the app's memory retention. The example demonstrates how to use Nebula, a conversational LLM created by Symbl AI, in place of OpenAI's GPT-3.5.

Singapore-based tech company Mozat has developed an innovative wardrobe management approach with its app, Stylepedia. The app is designed to redefine how users engage with fashion by integrating Milvus, an open-source vector database, to power its smart image search system. This integration allows Stylepedia to manage a rapidly growing database of clothing images, respond to user queries in milliseconds, and handle user-uploaded photos with varying resolutions. By leveraging Milvus, Stylepedia offers personalized style recommendations, facilitates user connections, and enables image searches for similar clothing items.

Milvus 2.3.4, the latest update of the vector database platform, introduces enhancements to improve availability and usability. The release focuses on streamlining monitoring, data import, and search efficiency. Key highlights include access logs for improved system performance insights, Parquet file support for efficient large-scale data operations, Binlog index on growing segments for faster search within expanding datasets, and other improvements such as increased collection/partition support, enhanced memory efficiency, clearer error messaging, faster data loading speeds, and better query shard balance. Developers are encouraged to visit the release notes for a comprehensive overview of all new features and enhancements in Milvus 2.3.4.

Distributed systems are crucial for vector search applications, offering scalability, fault tolerance, enhanced performance, and global accessibility. Consistency is a key principle in distributed systems, ensuring that data remains accurate across all replicas. The fully distributed Milvus vector database offers Tunable Consistency through its unique architecture, allowing users to scale out data writing while maintaining consistency without additional tools. Consistency levels in Milvus include Eventual, Session, Bounded, and Strong. Eventual consistency ensures that data will eventually be consistent across all replicas, prioritizing speed over immediate data updates. Session consistency maintains up-to-date data within a single session, while Bounded Consistency forces instances and replicas to sync within a certain period. Strong consistency ensures immediate data availability but comes with increased latency. Understanding the levels of consistency is essential for building resilient, high-performing applications that utilize distributed systems.

OpenAI's built-in retrieval feature has storage constraints, performance gaps, and cost concerns. The current pricing model of $0.2 per GB per day is expensive compared to traditional document services like Office 365 and Google Workspace. However, the server cost for serving vectors is only about $0.30 per day, which is a bargain compared to the pricing. The architecture of OpenAI Assistants' retrieval feature has limitations such as a maximum of 20 files per assistant, a cap of 512MB per file, and a hidden limitation of 2 million tokens per file. The current architecture may not scale well enough to support larger businesses with more extensive data requirements. To address these challenges and reduce costs, the service's architecture needs to be optimized. A refined vector database solution, hybrid disk/memory vector storage, streamlining disaster recovery by pooling system data, and multi-tenancy support for diverse user base are suggested improvements. Among popular vector databases, Milvus is considered the most mature open-source option with effective separation of system and query components, isolation of query components through Resource Group feature, hybrid memory/disk architecture, and application-level multi-tenancy facilitated by RBAC and Partition features. However, no single vector database solution can comprehensively address all challenges and meet every design requirement for imminent infrastructure development. The choice of vector databases should be tailored to specific requirements to effectively navigate the complexities of optimizing OpenAI Assistants' architecture.

The OpenAI Assistants' retrieval feature has been a topic of discussion in the AI community, as it incorporates Retrieval Augmented Generation (RAG) capabilities for question-answering. A comparison between OpenAI's built-in RAG and a customized RAG using Milvus shows that while the former slightly outperforms in answer similarity, the latter performs better in context precision, faithfulness, answer relevancy, and correctness. The Milvus-powered Customized RAG system also has higher Ragas Scores than OpenAI's built-in RAG. This superior performance is attributed to factors such as effective utilization of external data, better document segmentation and data retrieval, and the ability for users to adjust parameters in the customized RAG pipeline.

The blog post discusses the disparities between benchmark results of Qdrant's vector database technology and VectorDB Bench, which uses Milvus. It highlights three reasons for these differences: outdated Milvus version used in testing, improper use of Milvus by only using Growing Segments, and benchmark-driven optimizations for Qdrant that may compromise operational flexibility in real-world scenarios. The post emphasizes the importance of trustworthy and comprehensive benchmarking for vector databases and suggests developers should access truthful and precise benchmarks or conduct their own tests against their data to make informed decisions when choosing a vector database.