Introducing TensorFlow support

Vespa has introduced a new feature allowing the direct import and deployment of TensorFlow models to enhance machine-learned ranking capabilities, simplifying the previous cumbersome process of converting models to Vespa’s tensor format. This feature enables parallel execution of models across multiple threads and machines, ensuring efficient evaluation of large data items while maintaining bounded response times. Vespa optimizes model evaluation by extracting parameters into Vespa tensors and generating efficient tensor expressions, avoiding the overhead associated with TensorFlow’s inference engine. Users can import models using the SavedModel API and integrate them into the Vespa application package, specifying input through macros. Although the current support includes only a subset of TensorFlow operations, with limitations on complex models like convolutional and recurrent networks, Vespa plans to expand functionality and improve performance, with future support for additional frameworks such as ONNX.