Qdrant Summer of Code 2024 - ONNX Cross Encoders in Python

During Qdrant Summer of Code 2024, Huong (Celine) Hoang added ONNX-based cross-encoder reranking support to FastEmbed, enabling applications to score query-document pairs and reorder results retrieved from Qdrant. Unlike embedding models that convert text into vectors, cross-encoders evaluate a query and candidate document together to produce relevance scores, requiring new base and ONNX encoder interfaces, paired-input tokenization, token type ID handling, batching, and model-specific output processing. The implementation supported models including Xenova MS MARCO MiniLM variants and BAAI rerankers while avoiding PyTorch or TensorFlow dependencies through ONNX Runtime. Accuracy was validated by comparing FastEmbed’s ONNX scores with corresponding PyTorch outputs under a defined tolerance, with tests using example queries and documents such as Paris and Berlin. Released in FastEmbed 0.4.0, the project highlighted the importance of testing the complete developer experience beyond successful model loading, while identifying future work in broader model, tokenizer, and batch-processing support.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.