March 2023 Summaries

Milvus, an open-source vector database, has released version 2.2.5 with new features and improvements. Key updates include a security fix for MinIO (MinIO CVE-2023-28432) by updating to the latest release, and the addition of a First/Random replica selection policy that selects replicas in a round-robin fashion, improving throughput. The release also includes bug fixes and performance enhancements. For more information, check out the release notes or download Milvus to get started.

OpenAI has open-sourced the code for a knowledge base retrieval plugin, allowing ChatGPT to augment its information by retrieving knowledge-based data from relevant document snippets. The plugin uses OpenAI's text-embedding-ada-002 embeddings model and stores the embeddings into a vector database like Milvus or Zilliz. Enterprises can benefit from this plugin by making their internal documents available to employees through ChatGPT, ensuring accurate and up-to-date information retrieval. The plugin also supports continuous processing and storage of documents from various data sources using incoming webhooks. Additionally, the memory feature allows ChatGPT to remember information from conversations and store it in a vector database for later use.

Milvus is an open-source vector database that supports eight types of Indexes to optimize data querying and retrieval. These include FLAT, IVF_FLAT, IV_SQ8, HNSW Index, IVF_PQ, ANNOY, BIN_FLAT, and BIN_IVF_FLAT. Each Index type is best suited for specific scenarios based on factors such as data dimensions, dataset size, search efficiency requirements, and available resources. Choosing the right Index type can significantly improve search performance in AI applications.

The Beta release of Milvus 2.3 introduces new features and improvements aimed at boosting the performance of AI-powered applications. Key features include support for GPU acceleration, RAFT-based integration, range search capabilities, mmap file I/O, incremental backups, and change data capture (CDC). These enhancements enable faster and more efficient vector data searches, improved productivity, and better overall performance of AI systems. The release also includes bug fixes and improvements for a smoother user experience.

Developers often ask how Milvus compares to previous versions for embedding workloads, with concerns about performance degradation. Benchmarks conducted on Milvus v2.2.3 vs. v2.2.0 and v2.0.0 show that the latest version significantly improves search and indexing speeds. Specifically, Milvus 2.2.3 achieved a 2.5x reduction in search latency compared to the original Milvus 2.0.0 release and a 4.5x increase in QPS. The performance evaluation technical paper provides detailed methodology and results. Periodic re-running of benchmarks will update the findings, with all code available on Github for further verification or suggestions.

Milvus 2.2.4 has been released, featuring resource grouping for QueryNodes to improve performance and better manage resources in multi-tenant scenarios. Additionally, enhancements include collection renaming, Google Cloud Storage support, and a new option (ignore_growing) for search and query APIs. The release also includes bug fixes and performance improvements. For more information, check the release notes or download Milvus to get started.

The text discusses the importance of data protection, security, and availability when moving vector search workloads to the cloud. It highlights three pillars of information security - confidentiality, integrity, and availability. The text also mentions common data management mistakes and how Zilliz Cloud offers features to protect users' data and services by ensuring confidentiality, integrity, and availability.