August 2022 Summaries

In this post, the author demonstrates how to build a semantic similarity search engine for scientific papers using open-source tools like ArXiv, Dask, sentence-transformers, and Milvus vector database. The process involves setting up an environment, downloading the arXiv dataset from Kaggle, loading data into Python using Dask, implementing a scientific paper semantic similarity search application using Milvus vector database, and running queries to find similar papers. This approach can be used as a template for building any NLP semantic similarity search engine, not just scientific papers. The author also provides an overview of the SPECTRE model, which is used to convert texts into embeddings.

Zilliz Cloud, a fully-managed vector database cloud service built around Milvus, has been launched in preview mode for early access application. The service is designed to manage and process feature vectors at scale and in real-time, addressing the needs of modern AI algorithms that represent the deep semantics of unstructured data with feature vectors. Zilliz Cloud supports much-desired Milvus features while relieving users from managing their own data infrastructure. The service is designed for enterprise-level AI development and offers a fully-managed experience, high performance, elastic deployment, and enterprise-level security. Currently in private preview, interested parties can apply for early access by filling out a form on the Zilliz website.