Lattice: an 8 MB static retriever that embeds Wikipedia in 7 minutes

Lattice-retrieval is a static sentence-embedding model consisting only of a learned token lookup table, mean pooling, and normalization, designed to prioritize compactness and throughput over transformer-level contextual understanding. Trained on roughly 660 million curated query-document pairs and then fine-tuned with hard negatives, it achieved 0.4581 NDCG@10 on decontaminated BEIR before fine-tuning and 0.4749 afterward, surpassing the cited static-retrieval reference model. The project used a custom high-throughput training pipeline based on pre-tokenized, memory-mapped binary data that sustained approximately 360,000–370,000 pairs per second on four A100 GPUs. Experiments found that static embedding tables tolerate quantization especially well: a 512-dimensional int4 model using per-row scales retained nearly full-precision retrieval quality in a 7.94 MB file, while per-dimension scaling was faster but could lose low-magnitude token information at low precision. A pure-Rust SIMD runtime then embedded all 6.4 million English Wikipedia articles in 7 minutes and 26 seconds on an 8-core Apple M2 MacBook Air, processing 9.52 million tokens per second, with tokenization accounting for most runtime. The work concludes that although the model is not a replacement for strong transformer retrievers, it is a practical option for low-cost first-stage retrieval, clustering, deduplication, hard-negative mining, and on-device or browser deployment.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.