December 2019 Summaries

In December 2019, Vespa announced several product updates aimed at enhancing its open-source big data processing and serving engine, which is widely used in Yahoo products and the Verizon Media Ad Platform. Key enhancements include improved support for ONNX models, specifically adding operations like General Matrix to Matrix Multiplication and better PyTorch integration, alongside the introduction of a new ranking feature called attributeMatch().maxWeight. Additionally, Vespa implemented free lists for multivalue attribute mapping to boost performance by reducing CPU usage and memory, particularly beneficial for applications with high update rates. Other improvements include faster updates for out-of-sync documents using bucket checksums and the adoption of Apache ZooKeeper 3.5.6 with encrypted communication capabilities. These updates reflect Vespa's ongoing development and its commitment to integrating community feedback and contributions.

Vespa.ai has published two tutorials designed to help users get started with text search applications by leveraging the capabilities of Vespa, a platform for building scalable search solutions. Based on Microsoft's MS MARCO dataset, the first tutorial guides users through creating a basic text search application, deploying it with Vespa, and experimenting with various ranking functions like nativeRank and BM25. The second tutorial focuses on generating training datasets to enhance app ranking functions, emphasizing the importance of listwise loss functions over pointwise ones for optimizing Mean Reciprocal Rank (MRR), using frameworks like TF-Ranking. The tutorials provide detailed instructions and code to facilitate the reproduction of steps, encouraging users to experiment with different ranking profiles and improve search result relevance.