February 2024 Summaries

Zilliz has introduced Zilliz Cloud Bring Your Own Cloud (BYOC) to provide greater data sovereignty and compliance. This solution allows customers to use Zilliz Cloud's managed services while keeping their data within their private network. The architecture of Zilliz Cloud BYOC is built on two main pillars: the Data Plane, which encompasses all essential components for data collection, management, and query processing; and the Control Plane, responsible for deployment, management, and seamless coordination across all instances of the Zilliz Data Plane. The new deployment model in BYOC lets customers deploy the Data Plane within their own Virtual Private Cloud (VPC) while the Control Plane remains managed by Zilliz. This setup offers benefits such as data security and compliance, fine-grained control, and cost savings. Security measures include adherence to the Principle of Least Privilege, controlled access for software updates, and data plane access restrictions. BYOC is currently available on AWS with plans to expand to other cloud providers in the future.

Sohu, a NASDAQ-listed company, partnered with Milvus to enhance its news recommendation system. The outdated legacy vector search stack in the recommender system was struggling to deliver real-time, personalized news due to slow retrieval and scalability issues. Milvus, an open-source vector database, provided a solution for handling large datasets and improving classification accuracy of short-text news articles. Sohu News integrated Milvus into its recommender system using a dual-tower structure and achieved a 10x faster vector retrieval speed and significantly improved recommendation accuracy. The collaboration with Milvus has transformed the user experience by offering more personalized and engaging news content.

This blog post discusses the use of embedding models in Retrieval Augmented Generation (RAG) applications. RAG is an approach used to enhance question-answering bots by integrating domain knowledge into AI's knowledge base. The process involves using embedding models to generate vector embeddings of chunks of text from all documents, followed by indexing and search using the same embedding model. Finally, a large language model (LLM) generates an answer based on the given domain knowledge. The most common type of embedding model is SBERT (Sentence-BERT), which specializes in understanding complete sentences. The HuggingFace MTEB Leaderboard provides a list of embedding models sorted by retrieval performance, making it easier for developers to choose the best model for their needs. Zilliz Cloud Pipelines support various embedding models, including BAAI/bge-base-en(or zh)-v1.5, VoyageAI's voyage-2 and voyage-code-2, and OpenAI's text-embedding-3-small(or large). Each model has its advantages and is best suited for different use cases. In conclusion, embedding models play a crucial role in enhancing AI retrieval capabilities by integrating domain knowledge into the AI's knowledge base. The choice of an appropriate embedding model depends on factors such as context length, embedding dimensions, and specific use case requirements.

This blog discusses building Retrieval Augmented Generation (RAG) applications without using OpenAI's GPT models. The authors demonstrate how to build RAG apps with Mixtral, Milvus, and OctoAI. They also provide an overview of the tools involved in this process: Mixtral as the LLM, Milvus as the vector database, OctoAI for serving the LLM and embedding model, and LangChain as the orchestrator. The tutorial covers setting up RAG tools, loading data into a vector database, querying data with OctoAI and Mixtral, and leveraging Mixtral's multilingual capabilities.

This tutorial explores the concept of multimodal embeddings using open-source tools like Voxel51 and Milvus. It covers the meaning of "multimodal", how Milvus handles multimodal embeddings, examples of multimodal models, and how to use FiftyOne and Milvus for multimodal embedding exploration. The tutorial uses Fashion MNIST dataset with CLIP-VIT model from OpenAI to demonstrate the process. It also discusses how to further customize FiftyOne for data exploration with Milvus and provides a summary of exploring multimodal embeddings with Voxel51 and Milvus.

The article details the creation of Zilliz Cloud, a fully managed service powered by Milvus, the most adopted open-source vector database, developed from the ground up over eighteen months. It covers the design choices and invaluable insights gained during the journey to build this cloud service. The author emphasizes maximizing the use of mature third-party products, simplifying architecture, anticipating day 2 challenges from day 1, and focusing on cloud finops as key principles for building a successful cloud service. They also discuss the lessons learned while creating a scalable Vector Search Service on the public cloud and acknowledge the support of their users in this endeavor.

A recent regression in Milvus has caused an error when using Langchain v0.1.5 to connect with it, specifically a "KeyError: 'pk'" error due to the absence of an automatically generated primary key field. The temporary solution is to downgrade to Langchain version <= v0.1.4 until the fix is officially merged. A permanent solution will be provided in an upcoming update that addresses this issue by handling cases where "pk" is not present during insertion. Until then, users can workaround the problem by downgrading their Langchain version or waiting for the official fix.

Zilliz has released an update to its Cloud Pipelines, focusing on embedding models and usability improvements. The February release includes new 3rd-party embedding models from OpenAI and Voyage AI, providing a total of six options for users. Additionally, the platform now supports all dedicated vector db clusters in GCP's us-west-2 region, enhancing performance and reliability. Usability improvements include a new "Run Pipeline" page, local file upload feature, and support for running pipelines on any type of vector database cluster.

Zilliz has joined the AI Alliance, a consortium promoting open innovation in AI for responsible development and safe practices. The company's journey into open source began with its vector database Milvus, which was donated to the Linux Foundation. Open-source projects foster transparency, collaboration, innovation, and accessibility. Zilliz is committed to working alongside other industry players within the AI Alliance to shape a future where AI benefits everyone and positively impacts society.

This integration enables developers to effortlessly transfer data from Spark/Databricks to Milvus/Zilliz Cloud, whether in real-time or batch mode. By leveraging the Databricks Connector for Apache Arrow, developers can streamline their workflow and focus on building efficient and scalable AI solutions using these powerful technologies. The integration approach involves connecting Spark to Milvus through a shared filesystem such as S3 or MinIO buckets. By granting access to Spark or Databricks, the Spark job can use Milvus connectors to write data to the bucket in batch and then bulk-insert the entire collection for serving. To help developers get started quickly, we have prepared a notebook example that walks them through the streaming and batch data transfer processes with Milvus and Zilliz Cloud. This integration empowers developers to build efficient and scalable AI solutions, unlocking the full potential of these powerful technologies. For more information on this integration and its use cases, check out the official documentation for Databricks Connector for Apache Arrow.

Zilliz Cloud, a fully managed Milvus vector database that supports various AI applications, is now available on Google Cloud Marketplace. This integration simplifies billing as charges will appear directly on the developer's regular Google Cloud bill. Users can easily subscribe to the Zilliz service using their existing GCP account and access all features without upfront costs. A 100 credit bonus is available for new sign-ups, enabling developers to kickstart their journey with Zilliz Cloud on GCP.

Zilliz Cloud Pipelines has integrated the Voyage AI embedding models, voyage-2 and voyage-code-2, which have shown outstanding performance in retrieval tasks related to source code, technical documentation, and general tasks. The incorporation of these models enhances the RAG system implemented with various embedding models for code-related tasks. Notably, when compared to other popular embedding models on code datasets, Voyage's models demonstrate significantly better retrieval capability and lead to over ten percentage point improvements in Answer Correctness and overall performance scores.

Zilliz Cloud has introduced enhanced Role-Based Access Control (RBAC) functionality, providing more nuanced RBAC capabilities to improve access management and data isolation. The updated system features two primary categories of roles - Operation Layer Roles and Data Layer Roles - catering to diverse developer requirements. In the operational layer, Zilliz Cloud has four predefined Organization and Project Roles: Organization Owner, Organization Member, Project Owner, and Project Member. Additionally, it offers three predefined Cluster Roles in the data layer: Admin, Read-Write, and Read-Only. Users can also create custom roles to fine-tune permissions for specific collections or operations. The enhanced RBAC capabilities are exemplified through real-world use cases such as cross-team collaboration in a medium-sized company and managing a RAG-based knowledge base. These features ensure effective data management, improved security, and efficient resource allocation.

In this comparison, we delve into the functionalities and performance of two open-source vector databases: Milvus and Chroma. We assess these platforms based on their capabilities in handling vector data storage, indexing, searching, scalability, and ecosystem support. Additionally, we examine the purpose-built features and performance trade-offs between Milvus and Chroma. Milvus is a versatile and comprehensive open-source vector database, offering extensive support for various index types, including 11 different options. It supports hybrid search operations and offers flexible in-memory and on-disk indexing configurations. Furthermore, Milvus ensures strong consistency and provides multi-language SDKs encompassing Python, Java, JavaScript, Go, C++, Node.js, and Ruby. On the other hand, Chroma is a relatively simpler vector database with a primary focus on enabling easy initiation and usage. It currently supports only the HNSW algorithm for its KNN search operations and lacks advanced features such as RBAC support. Additionally, it offers limited SDK options, primarily focusing on Python and JavaScript. While Chroma's simplicity may be adequate for specific applications, its limitations could restrict its adaptability across diverse use cases. With its comprehensive functionality and extensive feature set, Milvus emerges as a more versatile and scalable solution for addressing a broader spectrum of vector data management needs. In the upcoming Milvus 2.4 release, we plan to support the inverted index with tantivy, which promises significant enhancements to prefiltering speed. This update further solidifies Milvus as a cutting-edge open-source vector database that continues to evolve and adapt to emerging requirements in the AI ecosystem. In summary, while Chroma offers simplicity and ease of use, Milvus distinguishes itself with its comprehensive feature set, extensive index type support, and robust multi-language SDKs. As a result, Milvus remains a highly recommended open-source vector database for developers and organizations seeking to optimize their applications' performance, scalability, and data management capabilities. Milvus Lite, a lightweight alternative to the full Milvus version, has also been introduced. It aims to preserve the ease of initiation while retaining an extensive set of features, making it particularly useful for specific use cases such as integration into Python applications without adding extra weight or spinning up a Milvus instance in Colab or Notebook for quick experiments.

Milvus is a distributed system designed to scale vector operations, addressing the challenges of scalability in vector databases. Unlike traditional databases, vector data doesn't require complex transactions and has diverse use cases that necessitate tunable tradeoffs between performance and consistency. Some vector data operations are computationally expensive, requiring elastic resource allocation. Milvus achieves horizontal scaling through its deliberate design as a distributed system, overcoming the limitations of single-instance databases. It contains four layers: access, coordination, worker, and storage. The separation of concerns in querying, data ingestion, and indexing allows for independent scaling of each operation. Milvus ensures large-scale write consistency through sharding and supports pre-filtering metadata search to enhance efficiency. Its unique architecture provides benefits such as horizontal scaling and flexibility, making it a suitable choice for cloud-native vector databases catering to diverse use cases.

Cardinal is a new vector search engine developed by Zilliz, which has demonstrated a threefold increase in performance compared to the previous version. It offers a search performance (QPS) that reaches tenfold that of Milvus. Cardinal is capable of performing brute-force search, creating and modifying ANNS indices, working with various input data formats, and filtering results during the search based on user-provided criteria. The key to Cardinal's speed lies in its algorithm optimizations, engineering optimizations, low-level optimizations, and AutoIndex feature for search strategy selection.