July 2021 Summaries

In July 2021, Vespa.ai announced several key updates aimed at improving efficiency and functionality. HTTP/2 support was introduced for search and feed endpoints, enhancing network usage and security, and allowing efficient use of the document/v1 REST API alongside the Vespa HTTP client. Additionally, the integration of ONNX RUNTIME into the Vespa container enables the use of ONNX models with automatically generated REST APIs for stateless model serving, lightweight request handlers, and model evaluations for query processing, content transformation, and ranking phases. The Vespa Factory, an automated build and test system now open to the public, allows users to track performance improvements and inspect release changes. Vespa's capabilities, including approximate nearest neighbor filtering and real-time updates, were highlighted at the Berlin Buzzwords 2021 conference, showcasing its application in various fields. Developed largely by Yahoo engineers, Vespa is a versatile open-source big data processing engine used by several Yahoo products and the Verizon Media Ad Platform, continuing to grow with community feedback and contributions.

Vespa.ai enhances its capabilities by accelerating stateless model evaluation in its container cluster using ONNX Runtime, allowing for efficient processing and transformation of documents or queries before storage or execution. This development introduces new use cases, such as generating vector representations for natural language text and enabling nearest neighbor retrieval, which were traditionally performed in the content cluster. The Vespa platform supports both stateless and stateful model evaluations, with container nodes handling stateless evaluations that involve single model evaluations per query or document, while content nodes manage stateful evaluations that require multiple evaluations using combined query and document data. The platform's architecture allows for ease of deployment and flexibility, automatically enabling a REST API for model discovery and evaluation and facilitating the integration of custom request handlers, document processors, and searchers, which leverage machine-learned models to enhance query processing and document transformation tasks. The introduction of stateless model evaluation acceleration in Vespa.ai's container cluster further solidifies its position as a robust platform for executing low-latency computations over large datasets without the need for external model servers, thereby reducing system complexity and improving scalability.

Vespa, a search and recommendation engine, has integrated HTTP/2 support to enhance performance and efficiency, particularly for search and feed endpoints. HTTP/2 addresses performance limitations of HTTP/1.1 by enabling features like header compression and multiplexing, which allow for more efficient network usage and reduced latency. It also improves security by requiring newer versions of Transport Layer Security (TLS). With HTTP/2, Vespa can handle significantly more concurrent requests with lower CPU utilization, achieving up to 225,000 requests per second under optimal conditions. The inclusion of HTTP/2 allows for efficient data feeding through the /document/v1 REST API, which was previously less capable than Vespa's internal custom-protocol API. The new protocol facilitates high-throughput asynchronous operations and simplifies the implementation of feed clients. Vespa's HTTP/2 support is enabled by default on version 7.425 or newer, and users are encouraged to configure their systems to take advantage of this upgrade for better performance and throughput.