Introducing ONNX support
Blog post from Vespa
Vespa, an open-source big data serving engine, has introduced support for the Open Neural Network eXchange (ONNX) format, allowing for seamless evaluation of machine learning models from various frameworks with low latency over large datasets. This integration complements Vespa's existing TensorFlow support and enables interoperability among popular deep learning frameworks like TensorFlow, PyTorch, and MxNet by using ONNX as a common intermediate representation in model deployment and development. Vespa optimizes model evaluation for real-time responses across large datasets and aims to streamline the process of deploying models trained on different frameworks. Importing ONNX models in Vespa involves adding the model to the application package and referencing it using a new ONNX ranking feature. While Vespa currently supports many ONNX operations, some complex operations like convolutional neural networks and recurrent networks are not yet fully supported due to computational costs. Vespa continues to enhance its performance and expand ONNX support, including the ONNX-ML extension, and invites feedback from users to further improve its capabilities.