Filterable HNSW

Andrei Vasnetsov discusses the challenges and potential solutions for applying constraints during approximate nearest neighbor (ANN) searches in vector spaces, specifically focusing on the Hierarchical Navigable Small World (HNSW) algorithm. While traditional libraries like Annoy, FAISS, or NMSLib offer fast approximate searches, they struggle with incorporating constraints such as category or label filtering directly into the search process. Vasnetsov proposes modifying the HNSW algorithm by applying filter criteria to graph nodes, allowing for category-specific connections and enhancing graph connectivity to maintain search efficiency. This approach is applicable to categorical filtering, numerical ranges, and geographical searches, with the potential for further optimization in real production systems using faster implementations like NMSLib. Through experiments and theoretical insights, Vasnetsov highlights the method's effectiveness, although challenges remain in ensuring connectivity across categories and optimizing for different filtering scenarios.