Qdrant Summer of Code 2024 - WASM based Dimension Reduction

Jishan Bhattacharya's internship at Qdrant during the Summer of Code 2024 focused on optimizing vector visualization performance using WebAssembly (WASM) and Rust, under the mentorship of Andrey Vasnetsov. The project aimed to enhance the visualization of high-dimensional vectors in Qdrant's vector database by implementing a WASM-based dimensionality reduction process using the t-SNE algorithm. Bhattacharya rewrote the existing JavaScript implementation in Rust and employed multi-threading, resulting in improved performance, although challenges remained with large datasets. Further optimizations included using a Vantage Point Tree for efficient nearest neighbor searches and incorporating the Barnes-Hut approximation for faster gradient calculations. Despite significant performance gains, the project faced bottlenecks that led to a shift in computing nearest neighbors server-side and improvements in data transfer and rendering techniques. The internship not only expanded Bhattacharya's technical skills but also deepened his passion for real-world performance optimization, setting the stage for future contributions to Qdrant's vector visualization capabilities.