SingleStore/Agile MRL Embedding Vector Search Blog

Matryoshka Representation Learning (MRL) is an innovative approach to embedding models that mimics the nesting structure of matryoshka dolls, allowing for significant reductions in memory usage while maintaining search accuracy. By utilizing vectors that contain smaller, complete embeddings within them, MRL creates efficient vector indexes that offer dramatic improvements in memory reduction, throughput, and search recall. The technique involves building sub-vector indexes using the first few dimensions of a full-size vector, enabling a two-stage search strategy where initial candidates are retrieved quickly and then re-ranked using the full vector dimensions for precision. This method contrasts with traditional models that require full vector indexing, often resulting in memory inefficiencies. Tested on datasets with up to 10 million rows, MRL shows memory reductions of 82%-93% and throughput increases up to 6.6 times, with recall closely mirroring that of full vector searches. The use of MRL in combination with SingleStore's F16 vector support provides a scalable solution for AI applications, optimizing performance without sacrificing accuracy.