Using Large ONNX Models with External Data in Vespa Embedders

Vespa has introduced a new feature in its embedders to automatically handle large ONNX models that use external data files, addressing a long-standing limitation due to the 2 GB protobuf format cap. This enhancement allows Vespa embedders to download external data files referenced by ONNX models, enabling the integration of models that were previously too large to use directly within Vespa's built-in systems. The update, available from Vespa version 8.544, supports models used in embedders such as hugging-face-embedder, colbert-embedder, and splade-embedder, with the ability to use private models by propagating authentication tokens when downloading external data. However, embedders are the only components that currently support models with external data, and there are still constraints regarding model usage in ranking expressions, requiring models to be self-contained and under 2 GB. Additionally, external data files must be located in the same directory or a subdirectory relative to the model URL, and further support for ONNX models with external data in ranking expressions can be requested through issue tracking.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.