5x Faster HNSW Vector Search with int8 Quantization

Harper 5.1 introduces int8 quantization for HNSW indexes, significantly reducing storage requirements and improving search throughput, albeit with a minor recall degradation of approximately 1%. This quantization scales float components to a signed int8 range, resulting in a storage reduction from 3,072 bytes to approximately 772 bytes per vector node, and also decreases p99 search latency from about 9 seconds to 0.5 seconds under concurrent load. The asymmetric approach ensures that while stored graph nodes are quantized, query vectors remain full-precision, preserving query accuracy. Additionally, the update introduces a per-query ef override, allowing users to adjust candidate set size for graph traversal dynamically, which enhances recall without extensive configuration. The int8 quantization is ideal for large vector collections with high concurrency, offering substantial performance improvements with minimal impact on recall, whereas smaller collections with lower concurrency can still utilize the float32 path without noticeable throughput differences.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.