June 2021 Summaries

Collaborations between open-source communities have led to the integration of Milvus, the world's most popular vector database, and JuiceFS, a high-performance distributed POSIX file system designed for cloud-native environments. JuiceFS is commonly used for solving big data challenges, building AI applications, and log collection. A Milvus cluster built with JuiceFS works by splitting upstream requests using Mishards to cascade the requests down to its sub-modules. Benchmark testing reveals that JuiceFS offers major advantages over Amazon Elastic File System (EFS), including higher IOPS and I/O throughput in both single- and multi-job scenarios. The Milvus cluster built on JuiceFS offers high performance and flexible storage capacity, making it a valuable tool for AI applications.

Sohu, a NASDAQ-listed Chinese online media company, has built an intelligent news recommendation system inside its news app using semantic vector search. The system uses user profiles built from browsing history to fine-tune personalized content recommendations over time, improving user experience and engagement. It leverages Milvus, an open-source vector database built by Zilliz, to process massive datasets efficiently and accurately, reducing memory usage during search and supporting high-performance deployments. The recommendation system relies on the Deep Structured Semantic Model (DSSM), which uses two neural networks to represent user queries and news articles as vectors. It also utilizes BERT-as-service for encoding news articles into semantic vectors, extracting semantically similar tags from user profiles, and identifying misclassified short text. The use of Milvus has significantly improved the real-time performance of Sohu's news recommendation system and increased efficiency in identifying misclassified short text.