August 2024 Summaries

Jishan Bhattacharya's internship at Qdrant during the Summer of Code 2024 focused on optimizing vector visualization performance using WebAssembly (WASM) and Rust, under the mentorship of Andrey Vasnetsov. The project aimed to enhance the visualization of high-dimensional vectors in Qdrant's vector database by implementing a WASM-based dimensionality reduction process using the t-SNE algorithm. Bhattacharya rewrote the existing JavaScript implementation in Rust and employed multi-threading, resulting in improved performance, although challenges remained with large datasets. Further optimizations included using a Vantage Point Tree for efficient nearest neighbor searches and incorporating the Barnes-Hut approximation for faster gradient calculations. Despite significant performance gains, the project faced bottlenecks that led to a shift in computing nearest neighbors server-side and improvements in data transfer and rendering techniques. The internship not only expanded Bhattacharya's technical skills but also deepened his passion for real-world performance optimization, setting the stage for future contributions to Qdrant's vector visualization capabilities.

Kern AI has developed a low-code platform that addresses inefficiencies in AI model development by allowing developers to analyze datasets and identify outliers using vector search, thus enhancing data accuracy and workflow efficiency. With a focus on the financial services industry, Kern AI has integrated large language models, such as GPT, to create accurate and secure Generative AI solutions. Partnering with leading insurers, Kern AI has implemented AI chatbots to streamline customer service processes, significantly reducing response times and improving customer satisfaction. Their open-source tool, Kern AI Refinery, is used to scale and maintain natural language data, and their collaboration with Qdrant has been pivotal, leveraging its multi-vector storage and search capabilities for improved data handling and retrieval. This partnership has led to substantial achievements, such as maintaining a hallucination rate of under 1% in chatbots and reducing customer service response times at Markel Insurance SE, showcasing the platform's impact on operational efficiency in the finance and insurance sectors.

Qdrant leverages immutable data structures to enhance performance in vector search operations by optimizing for read-heavy workloads and hardware efficiencies. Immutability is beneficial as it allows for predictable memory allocation, reduced cache misses, and improved data retrieval speeds by using techniques like scalar quantization and perfect hashing. Despite challenges in updating data, Qdrant manages this by employing a segment-based storage system where new data is written to mutable segments and later converted to immutable ones. This approach, combined with methods like defragmentation, significantly boosts read operations' efficiency, though it acknowledges that immutable structures can increase update costs and require occasional index rebuilding. The architecture's flexibility allows users to adapt the system based on specific workload demands, balancing the advantages of immutability with the need for some level of mutability in data storage.

Dense embedding models, traditionally used for single-vector representations, can be effectively adapted for late interaction scenarios by leveraging their output token embeddings as multi-vector representations. This adaptation, facilitated by Qdrant's multi-vector feature, has shown that these models can compete with or even surpass specialized late interaction models in retrieval performance while offering reduced complexity and increased efficiency. Experimental results demonstrate that models like BAAI/bge-small-en outperform both sparse and late interaction models, although they require more storage due to higher-dimensional output token embeddings. Implementing vector compression can mitigate storage and computational costs without significantly affecting retrieval quality. Additionally, the new Query API in Qdrant 1.10 allows for sophisticated retrieval pipelines, enabling initial retrieval with pooled single vectors and subsequent reranking using output token embeddings. This approach promises enhanced retrieval quality and efficiency, highlighting the potential of dense models in advanced search applications while offering opportunities for further research and optimization.

Qdrant 1.11 introduces several enhancements aimed at optimizing data structures for improved scalability and efficiency, including better memory use, optimized segments, and on-disk payload indexing. This version offers features such as multitenancy support, which allows data for multiple users or regions to be managed more effectively by isolating tenant data and optimizing storage through defragmentation. The update introduces new Web UI tools like a Search Quality Tool for real-time semantic search testing and a Graph Exploration Tool for visualizing vector searches. The Query API is enhanced with capabilities like GroupBy Endpoint for grouping results, Random Sampling for selecting random subsets of data, and the addition of Distribution-Based Score Fusion (DBSF) for fusing hybrid query results. Additionally, the introduction of UUIDs as a new index type reduces memory usage by storing them as raw bytes, leading to significant memory savings. Overall, these updates are designed to manage large datasets efficiently, enhance query performance, and provide a more interactive user experience.