May 2024 Summaries

Milvus Lite, a lightweight and in-memory version of the Milvus vector database, is now available for easy installation and integration with Voyage AI embeddings, streamlining the development of generative AI (GenAI) applications. This tool is designed to be run on various platforms such as Jupyter Notebooks, laptops, or edge devices and provides robust semantic search capabilities through Voyage’s high-performing embedding models, which excel in the Massive Text Embedding Benchmark (MTEB) leaderboards. These models, including the general-purpose voyage-large-2-instruct and the legal-specific voyage-law-2, consistently outperform commercial alternatives like OpenAI and Cohere. Milvus Lite enables seamless scaling to production environments using the same client-side code for more scalable Milvus on Kubernetes or managed Milvus on Zilliz Cloud, thus simplifying migration and saving time. A demonstration showcases how Milvus Lite and Voyage embeddings facilitate semantic search by embedding documents and queries for efficient information retrieval.

Voyage introduces rerank-1, an advanced reranker that surpasses previous models like bge-reranker-v2-m3 and Cohere’s rerank-english-v3 and rerank-multilingual-v3 across 37 domain-specific and 50 multilingual datasets, including languages such as French, German, Japanese, Korean, and Spanish. This new model offers an 8k context length, doubling that of rerank-lite-1 and Cohere’s rerank-english-v3, enhancing the relevancy of existing search systems. The evaluation of rerank-1 involved a wide array of datasets in fields such as technical documentation, code, law, finance, medicine, and conversations, demonstrating its superior performance in retrieving relevant documents using both lexical and embedding-based search methods. The model is particularly effective in improving retrieval quality across various domains and languages, showing consistent superiority over other rerankers in tests and enhancing retrieval quality in cases where first-stage search methods alone were insufficient. Voyage encourages users to integrate rerank-1 into their existing search systems to boost retrieval quality and invites interest in their upcoming domain-specific and fine-tuning embeddings.

Voyage AI's latest text embedding model, voyage-large-2-instruct, has achieved the top position on the Massive Text Embedding Benchmark (MTEB) leaderboard, surpassing models from OpenAI and Cohere in tasks such as retrieval, classification, clustering, and reranking. This model, featuring a 16K context window and instruction tuning, incorporates insights from the development of previous models and ranks first in five of the seven benchmarked tasks. Voyage AI offers a range of models, including domain-specific ones like voyage-law-2, which leads in legal retrieval tasks, and emphasizes the benefits of voyage-large-2-instruct for general-purpose embedding. The company is recognized for its innovative approach and leadership in embedding models, inviting users to explore and provide feedback on its offerings.