July 2025 Summaries

maxsim-cpu is a Rust-based Python package designed to accelerate MaxSim scoring for late-interaction retrieval models such as ColBERT and ColPali on CPUs. Unlike single-vector retrieval, MaxSim compares each query token with every token in candidate documents, producing millions of small similarity calculations that can create substantial CPU latency despite being efficient on GPUs. Built on libxsmm small-matrix multiplication routines and additional optimizations such as fused operations, variable-length document handling, and Apple Silicon-specific paths, the package reportedly reduces CPU scoring overhead for roughly 1,000 documents from 50–100 milliseconds with PyTorch to about 5 milliseconds in tested cases. It supports normalized embeddings through separate functions for fixed- and variable-length documents, is available through PyPI for AVX2-capable Linux systems and Apple Silicon Macs, and aims to make CPU-based multi-vector retrieval more practical for cost-sensitive, local, and latency-sensitive applications.