What is Vector Quantization?

Vector quantization is a data compression technique that reduces the memory footprint of high-dimensional datasets while preserving essential information, facilitating more efficient storage and faster search operations. This is particularly useful for handling large datasets, such as embeddings from providers like OpenAI, where the memory and processing demands are significant. The Hierarchical Navigable Small World (HNSW) index is one method for organizing these vectors, but it is computationally expensive due to its requirement for random reads and sequential traversals. Quantization helps by compressing vectors into smaller sizes, and three primary methods—Scalar, Binary, and Product Quantization—are employed to achieve this. Scalar Quantization reduces memory usage by mapping vectors to a range represented by smaller data types like int8. Binary Quantization further minimizes memory usage by converting vectors into binary representations, leading to substantial speed improvements. Product Quantization splits vectors into sub-vectors, each represented by a centroid from a codebook, providing high compression ratios. However, these methods may result in a loss of precision, which can be mitigated through techniques like oversampling and rescoring. By configuring storage options, such as moving original vectors to disk, and leveraging technologies like io_uring for efficient I/O operations, systems can optimize the balance between resource use and performance. Qdrant, a vector search service, supports these quantization methods, offering flexibility in switching between them and adjusting parameters to suit specific needs while maintaining the original vectors for high-accuracy rescoring when necessary.