August 2021 Summaries

In 2021, significant advancements were made in the database industry. Zilliz, a leading company in this field, shared its latest research progress and achievements at VLDB Workshop 2021. The company introduced Milvus, an open-source vector database developed with machine learning methods. Milvus is designed for handling massive feature vectors and provides a complete framework for vector data update, indexing, and similarity search. It has been widely used in artificial intelligence applications and its performance surpasses that of other products. The research team behind Milvus also presented their design concept for the 2.0 version, which includes cloud-native, log-as-data, and unified batch-and-stream processing features.

The research paper "HM-ANN: Efficient Billion-Point Nearest Neighbor Search on Heterogenous Memory" proposes a novel algorithm called HM-ANN for graph-based similarity search. This algorithm considers both memory heterogeneity and data heterogeneity in modern hardware settings, enabling billion-scale similarity search on a single machine without compression technologies. The paper discusses the challenges of existing approximate nearest neighbor (ANN) search solutions due to limited dynamic random-access memory (DRAM) capacity and presents HM-ANN as an efficient alternative that achieves low search latency and high search accuracy, especially when the dataset cannot fit into DRAM.