🥃 Distilling Tiny Embeddings

The article introduces the BERT Hash Embeddings series, a new set of models that generate fixed-dimensional vectors for tasks such as semantic textual similarity and text classification. These models, which include the bert-hash-femto, pico, and nano-embeddings, offer a compelling alternative to MUVERA's fixed-dimensional encoding with ColBERT models by requiring fewer parameters and storage space while maintaining competitive performance. The BERT Hash Embeddings models leverage a two-step knowledge distillation process to achieve high efficiency and are particularly notable for their potential use in edge and low-resource computing environments, as they allow data processing without the need to leave the device. The article highlights the success of the bert-hash-nano-embeddings model and suggests future exploration of sequential distillation to further compress large models effectively. Additionally, the company NeuML, which developed these models, offers AI consulting services and is working on a platform for hosted applications, emphasizing innovation in creating micro models tailored to specific use cases.