April 2021 Summaries

Thiago Martins' blog post on the Vespa platform outlines the development of a sentence and paragraph-level question answering (QA) application using Vespa's capabilities, integrating both sparse and dense ranking features. The tutorial uses the Stanford Question Answering Dataset (SQuAD) v1.1 to demonstrate the process, highlighting how to generate embeddings for questions and sentences to enable semantic search. It details creating and deploying a Vespa application that combines semantic (dense) and term-based (sparse) search, including the steps to organize data, generate embeddings, and utilize Vespa's schema and rank profiles for efficient information retrieval. The application showcases the integration of Vespa's ranking features, such as BM25 and semantic similarity, and supports both sentence-level and paragraph-level retrieval using a hybrid approach. The post concludes by suggesting further exploration using Facebook's Dense Passage Retrieval methodology to enhance retrieval-based QA systems.