November 2023 Summaries

The article explores the debate over the best way to store embeddings for vector search, weighing the benefits of dedicated vector databases against integrating vector capabilities into existing databases. It argues that while integrated solutions may seem appealing due to their convenience, dedicated vector databases offer greater flexibility, scalability, and optimization for vector search tasks, particularly in large-scale systems. The piece also highlights that search engines, often mislabeled as databases, prioritize scalability, speed, and availability differently than traditional databases, which are built on ACID principles. It notes that using a dedicated vector database can avoid performance issues and complex data synchronization problems associated with integrating vector features into a primary database. Ultimately, the article suggests that while simple vector search tasks may not require a specialized database, significant or central vector search functionalities would benefit greatly from a dedicated solution.

Cohere's Head of Machine Learning, Nils Reimers, discussed the evolution of embedding models at Cohere during the second edition of Vector Space Talks, highlighting several key developments. He emphasized the importance of content quality estimation in embeddings, which goes beyond traditional topic matching to differentiate between informative and non-informative documents. Nils also introduced compression-aware training to reduce the memory footprint of embeddings, making them more cost-effective for platforms like Qdrant. By applying reinforcement learning from human feedback, Cohere's models are able to learn preferences and produce more informative responses. He stressed the significance of evaluating embedding quality in relative terms, focusing on the context and relationships between embeddings. Additionally, Nils provided insights into upcoming features, such as input type support for Langchain and advanced compression techniques, while noting the challenge in differentiating true from fake statements due to reliance on pretraining data.