October 2020 Summaries

In the October 2020 Vespa product update, several advancements were introduced, including new container thread pool configurations and improvements to the document/v1 API throughput. The new configurations allow for live reloading of application changes into the JVM, enhancing feed throughput and minimizing latency during deployments by adjusting the default thread pool size to be proportional to logical CPUs instead of a fixed 500 threads. Optimizations in feed container concurrent execution and asynchronous request handling have boosted the quality of service and increased throughput by 50% for the synchronous HTTP API since Vespa version 7.304.50. Additionally, the previous visibility-delay feature for batching writes was removed due to inefficiencies, with recommendations to discontinue its use. Vespa, developed largely by Yahoo engineers, is an open-source big data processing engine utilized by platforms like Yahoo News and Verizon Media Ad Platform, and it continues to grow with community contributions.

In September 2020, Vespa introduced several updates enhancing its capabilities in processing and serving big data. A significant update was the integration of ONNX-Runtime into Vespa’s ranking framework, greatly improving the evaluation of large deep-learning models, specifically NLP models like BERT, with the aid of hardware acceleration and quantization for optimized performance. This update enabled the creation of a state-of-the-art question/answering system within Vespa that utilizes approximate nearest neighbors and large BERT models. Additionally, enhancements included support for the Hamming distance metric in ranking, improved performance for conditional writes, and a compressed transaction log that increases update throughput significantly. Vespa, developed by Yahoo engineers and used by platforms like Yahoo News and Verizon Media Ad Platform, continues to evolve with community feedback and contributions.