January 2024 Summaries

Qdrant's version 1.7 introduces a new Discovery Search API that enhances search functionality by allowing users to constrain searches within a vector space using context, which consists of pairs of positive and negative vectors. This approach allows for a controlled exploration of vector spaces, enabling searches to identify points that are relevant but not necessarily closest to a target. The API facilitates searches by partitioning the vector space into areas that favor positive over negative vectors, inspired by the triplet loss concept in machine learning. The Discovery Search can be applied in various real-world scenarios, such as searching for images using a multimodal encoder like CLIP, which embeds text and images into the same vector space, allowing for refined searches by excluding unwanted results. Context Search, a component of Discovery Search, enables exploration without a specific target, which can be useful for breaking out of recommendation bubbles in services like music streaming. By incorporating positive and negative feedback, users can refine searches to yield more diverse and controlled results.

Dr. Mikko Lehtimäki discusses the concept of the "bitter lesson" in AI, which suggests that data and computational capabilities will eventually outperform human-designed methods. His work at Softlandia focuses on retrieval-augmented generation (RAG) for language models, particularly through the development of Yokot AI, a productivity tool that integrates with enterprise data. Lehtimäki elaborates on the challenges of managing diverse data inputs and highlights the importance of a re-ranking component in improving the efficiency and quality of language model outputs. He underscores the complexity involved in designing RAG architectures and posits that future innovations in AI will likely enhance these systems, particularly the re-ranker, which plays a crucial role in optimizing data retrieval for language models.

Diptanu Gon Choudhury, founder of Tensorlake, has introduced Indexify, an open-source engine designed to create near-real-time knowledge bases from unstructured data, which aims to enhance AI-driven workflows and query engines. With a background in developing significant technologies at companies like Hashicorp, Netflix, and Facebook, Diptanu's experience informs his approach to reimagining data infrastructures, notably through Indexify's ability to integrate with existing data systems and streamline processes such as real-time indexing and data extraction. This innovation supports AI applications, including AI copilots for call centers, by enabling dynamic interaction with ever-evolving data sets. The development of Indexify is inspired by the increasing use of Large Language Models (LLMs) and seeks to address the challenges of modern data infrastructure, offering a more agile and practical solution compared to traditional platforms like Spark. Demetrios Brinkmann hosts a discussion with Diptanu, exploring the intricacies of Indexify and its potential impact on AI workflows, emphasizing the importance of efficient data management and retrieval-augmented generation (RAG) in contemporary tech environments.

In a recent discussion, Stanislas Polu, Co-Founder and Engineer at Dust, shared insights into his journey from Stripe to OpenAI, culminating in the creation of Dust, a company aimed at enhancing workplace productivity with large language models (LLMs). Polu elaborated on the strategic use of SaaS platforms and AI in enterprise data management and productivity enhancement, highlighting the importance of customized AI assistants in streamlining workflows. He emphasized the decision to use Qdrant for its performance and open-source nature, which supports their focus on integrating AI into enterprise settings effectively. Polu also discussed the challenges and strategies involved in product development and deployment, particularly the emphasis on creating specialized AI assistants tailored to specific tasks within companies, while navigating the complexities of enterprise data access and integration. His experience underscores the potential of AI in transforming business operations and the critical role of practical application and user engagement in realizing AI’s full potential.

Qdrant has announced a $28 million Series A funding round led by Spark Capital, with participation from existing investors Unusual Ventures and 42CAP, marking a significant milestone in its mission to develop a leading vector database. The funding aims to accelerate Qdrant's growth by enhancing its ability to handle complex, high-dimensional data, essential for next-generation AI applications, particularly as traditional keyword-based data retrieval methods fall short in the face of unstructured data like images, audio, and text. Built with efficiency, scalability, performance, and flexibility as core principles, Qdrant leverages Rust for enhanced performance and memory safety, and offers features such as a custom HNSW search algorithm, built-in compression, and multitenancy to cater to enterprise needs while emphasizing privacy and security. The rise of generative AI has highlighted the importance of vector databases, and Qdrant is positioned as a game-changer in this domain, offering hybrid SaaS solutions and open-source reliability to empower enterprises in building cutting-edge AI applications.

Qdrant has expanded its managed vector database service, Qdrant Cloud, to Microsoft Azure, enabling users to quickly set up and scale environments for rapid application development. This collaboration allows users to deploy clusters via the Qdrant Cloud Console in seconds, facilitating the handling of large-scale datasets with billions of vectors through Azure's scalable infrastructure. Features such as horizontal scaling and binary quantization are leveraged, providing robust support for enterprise-scale AI applications. Bosch Digital's AI team highlights Qdrant as a critical component in developing their provider-independent multimodal generative AI platform.

Qdrant's latest benchmark update for 2024 reveals significant improvements in the performance of vector search engines, achieved through the incorporation of user suggestions for better efficiency, resulting in up to four times the performance gains in certain tests. The benchmarks now include a new dataset of 1 million OpenAI embeddings to better align with the needs of RAG applications, and a clear distinction is made between latency and requests-per-second scenarios to accurately reflect diverse application requirements. Qdrant maintains its commitment to open-source principles, ensuring accessibility and fairness by limiting benchmarks to open-source solutions without external cloud influences, and continues to focus solely on vector databases rather than broader library or algorithm comparisons. Detailed results and opportunities for community involvement are available in their report and benchmark repository, inviting contributions to further enhance vector database performance.

The podcast on search technologies, particularly focusing on retrieval-augmented generation (RAG) in language models, explores how RAG combines information retrieval with language generation to enhance natural language processing capabilities. This approach allows AI to access external knowledge sources, leading to more accurate and contextually relevant outputs, thereby improving tasks such as question answering, summarization, and conversation setup. The discussion highlights the importance of evaluating RAG and large language models (LLMs) to ensure quality and incorporate feedback loops, addressing challenges like setting up expected document sets and measuring subjectiveness. Key evaluation aspects include model understanding at the domain level, data ingestion and processing strategies, retrieval precision, and generation guardrails. The podcast was organized by DataTalks.Club, and further discussions can be accessed through events organized by DeepRec.ai and resources like the Qdrant Blog.

Andrey Vasnetsov, CTO at Qdrant, emphasizes the importance of treating systems like Qdrant as search engines rather than traditional databases due to their focus on scalability and performance over transactional consistency. As a part of the Carnegie Mellon University Database Research Group's ML⇄DB Seminar Series, Vasnetsov delves into optimizing vector search by integrating in-place filtering during graph traversal, which enhances precision without compromising search accuracy even at large scales. He discusses strategies such as using overlapping intervals and geo hash regions to improve connectivity and precision within vector search indices, and highlights the differences between search engines and relational databases, stressing the significance of application needs in choosing the right system for scalability. Vasnetsov also explores techniques to control search precision and speed by adjusting the beam size in HNSW indices, and the challenges of maintaining a connected graph while filtering, along with innovative compression methods to optimize vector data handling.

Hamza Farooq, an AI enthusiast and founder of Traversaal.ai, discusses the future of Large Language Models (LLMs) and their application in creating complex and conversational search experiences. He highlights the potential of LLMs to transform how users interact with search engines by enabling more natural and personalized queries, which he demonstrates through examples like TripAdvisor's AI-generated travel itineraries. Farooq emphasizes the importance of deploying robust infrastructures to handle potential downtimes and the significance of user experience (UX) in developing GenAI products. Traversaal.ai aims to simplify the integration of LLMs into business operations by offering a unified platform that supports various applications such as chatbots and recommendation systems. He notes the growing demand for UX specialists to bridge the gap between technical and user-centric design aspects in AI-driven applications, highlighting their critical role in the evolving landscape of AI technology.

Filip Makraduli explores the use of AI to improve music recommendation systems by capturing and understanding the emotional "vibe" of songs through human language and AI models. His innovative approach uses large language models to generate song descriptions and employs sentence transformers to create embeddings, which are then analyzed in vector spaces using cosine similarity to make personalized song recommendations. Makraduli demonstrates the potential of this system with a live demo, showcasing its ability to create Spotify playlists tailored to specific moods. His work is part of a broader interest in leveraging AI for more nuanced and personalized user experiences, reflecting his expertise in data science and machine learning, particularly in unconventional recommender systems.

Loading data into Qdrant, a vector search engine, from Unstructured.io involves a series of steps starting with data extraction, cleaning, chunking, and generating embeddings before finally loading it into Qdrant. The blog post details the process of ingesting data from Discord channels into Qdrant using the Unstructured CLI, which supports over 20 vetted data sources. It outlines the prerequisites needed, such as a running Qdrant instance, a Discord bot token, and the Unstructured CLI with specific extras. The process involves generating structured data using a Discord bot, setting up Qdrant collections with specific vector dimensions, and loading the data with configurable options for embedding fields, partitioning, and chunking. The post also points to additional resources for setting up and configuring the Qdrant ingestion process, emphasizing Unstructured's capability to function programmatically or via a hosted API.

Noé Achache, a Lead Data Scientist at Sicara, shares insights on the complexities and applications of vector search, particularly in image search and retrieval-augmented generation (RAG) projects. He discusses the efficacy of Dino V2, a model developed by Meta for image representation, which has proven superior to traditional methods, highlighting its ability to understand objects without fine-tuning. Achache also delves into challenges such as data deduplication in real estate listings and the intricacies of document retrieval in multilingual and medical contexts. He emphasizes the importance of data safety, especially when dealing with sensitive medical information, and suggests that while fine-tuning is currently necessary for image search, text search can benefit from hybrid search techniques and better models. Achache notes the growing need for new models that address industrial demands, particularly those that can enhance image embedding with text guidance and automate document chunking. Throughout, he underscores the role of platforms like Qdrant in facilitating efficient and cost-effective vector database management.

Rishabh Bhardwaj, a data engineer at HRS Group, shared insights into building a high-performance hotel entity matching solution using Qdrant, an open-source vector database, during a Vector Space Talk with Demetrios Brinkmann. Initially, the project experimented with Postgres, but Qdrant was found to offer superior performance in terms of speed and recall, aided by the Hierarchical Navigable Small World (HNSW) algorithm. The solution addresses data inconsistency, duplication, and real-time processing challenges by employing the Mini LM model for embedding creation, which balances speed and accuracy effectively. Geofiltering is used to ensure accurate matching based on hotel locations, while GDPR compliance is maintained through secure infrastructure. The project evolved from an MVP using Postgres to a scalable architecture leveraging AWS services, demonstrating significant improvements in both performance and resource optimization.

In an insightful discussion, Andrey Vasnetsov, CTO of Qdrant, explores the transformative potential of binary quantization in vector space technology, highlighting its ability to significantly reduce storage size and enhance processing speed by up to 30 times. Despite its simplicity, binary quantization compensates for precision loss through oversampling, offering real-time accuracy adjustments without altering stored data structures. While compatible with certain models like OpenAI, the technique is not universally applicable, prompting ongoing research to identify the factors influencing model compatibility. Vasnetsov delineates the operational dynamics of Qdrant's vector search engine, focusing on the complexities of HNSW vector indexes and the necessity of quantization in addressing the growing demands of vector dimensionality. The conversation underscores the strategic balance between leveraging cutting-edge techniques and preparing for future advancements in data-intensive applications.

FastEmbed, a creation by Nirant Kasliwal, is a Python library designed to generate embeddings quickly and efficiently with a focus on production needs. Kasliwal, an AI engineer at Qdrant, highlights the library's capability to enhance CPU performance through quantized embedding models and discusses future plans for GPU-friendly quantized models. FastEmbed targets the challenges associated with embedding creation by providing a lightweight solution that prioritizes speed, efficiency, and accuracy without the overhead of training-time capabilities. The library, which supports multimodal embedding, is built to streamline the embedding process on local compute, allowing users to maintain control and simplicity in their workloads. Kasliwal's insights also cover practical tips and innovative methods for improving embedding models, including the use of linear layers and mixed precision embeddings for fast and cost-effective inference.

In a discussion about enhancing semantic search capabilities, Nicolas Mauti, an MLOps Engineer at Malt, detailed their transition to the Qdrant database to improve matching between freelancers and projects on their platform. This shift was necessitated by the platform's exponential growth, which introduced performance challenges such as increased latency. By adopting a retriever-ranker architecture and implementing multilingual transformer-based models, Malt significantly reduced latency from 10 seconds to 1 second, enhancing both performance and scalability. The decision to use Qdrant was influenced by its superior performance, precision trade-offs, and its capability to handle complex filtering requirements, including geospatial filtering. Mauti also highlighted the benefits of deploying Qdrant in a Kubernetes-based, GitOps-managed environment, which ensured robust and scalable operations. The result was a dramatic improvement in application latency, enabling Malt to develop more sophisticated matching models without compromising scalability.

Anush Shetty's blog post on building a chat service with a codebase using Qdrant and N8N outlines the process of creating AI-powered workflows with minimal coding. It highlights the use of N8N, an automation tool that connects apps via APIs, in conjunction with Qdrant, an open-source vector database, to ingest a GitHub repository and develop a chat service. The first workflow involves using nodes like Qdrant Vector Store and GitHub Document Loader to transform GitHub data into vector embeddings, which are stored in Qdrant, using OpenAI's text-embedding-ada-002 model. The second workflow retrieves these vectors to facilitate a chat service by employing nodes such as Qdrant Vector Store - Retrieve, Retrieval Q&A Chain, and OpenAI Chat Model, specifically utilizing the gpt-3.5-turbo model for interaction. The post also suggests embedding the chat service in applications using the @n8n/chat package and notes N8N's capability to support scheduled and event-triggered workflows.