Introducing NLP with Transformers on Vespa

NLP has undergone a significant transformation with the advent of Transformer models like BERT, which excel in various NLP tasks and have garnered widespread adoption due to the ease of use provided by resources like the Hugging Face Transformers library. Vespa, a distributed application platform, offers a unique approach to deploying these models by moving computation to data, thus reducing network bottlenecks and complexity. It enables efficient evaluation of machine-learned models, like Transformers, by automatically distributing them across content nodes, which handle data-dependent computations. This approach is particularly advantageous for search and recommendation applications that require evaluating numerous data points per query. Vespa's flexibility allows for combining different models and computations, such as using BM25 for initial ranking followed by a Transformer model for re-ranking, and it supports models from various platforms like TensorFlow and PyTorch. The platform's capabilities are demonstrated with a sample application using the MS MARCO dataset, showcasing the integration of a small Transformer model for text ranking while highlighting future improvements aimed at enhancing performance and model support.