My SIMD Is Faster than Yours

Eto's engineering team decided to rewrite the Lance columnar format in Rust to enhance the performance of SSD-resilient vector searches in LanceDB, leveraging Rust's capabilities like native code-generation and extensive CPU intrinsics support. The focus was on optimizing the computation of K-nearest-neighbors in high-dimensional vector spaces using different strategies. Initial attempts with naive L2 distance implementations and Apache Arrow compute kernels revealed limitations, primarily due to inefficient memory allocations and cache usage. The team then explored BLAS libraries and manual tuning of SIMD instructions for both x86_64 and Apple Silicon architectures, achieving significant speedups by utilizing Intel's AVX-2 and Apple's NEON instructions. Future improvements for LanceDB include adopting AVX-512 routines, supporting different data types, and introducing GPU acceleration. The Rust ecosystem, particularly tools like cargo and Compiler Explorer, proved invaluable for cross-platform optimization and benchmarking efforts, encouraging further exploration of I/O path optimizations for NVME SSDs and potential shifts to the std::simd project once it becomes available in stable Rust.