February 2022 Summaries

NLP resources are unevenly distributed across languages, with English being overrepresented. However, creating one's own annotated resources can be advantageous, allowing for full control over data selection and annotation. The Haystack framework offers a range of tools to create NLP resources, including preprocessing tools, an annotation tool, and a question generation node. Other smart people have devised crafty solutions for dataset creation in low-resource settings, such as translating existing datasets or using data augmentation techniques. Joining the Haystack NLP community can help collect more resources for underrepresented languages and connect with other users and the engineering team. Monolingual models perform better than multilingual models on certain tasks, suggesting a focus on creating monolingual NLP resources is beneficial. The creation of SQuAD-like datasets for German and French has shown improved model performance compared to multilingual models. A range of tools and techniques are available to create and preprocess NLP resources, making it more accessible to developers.

Haystack is introducing new tools for question answering on tables using its NLP framework. Structured data, such as tables, can be processed by computers due to their standardized format. The Haystack NLP framework allows users to extract answers from tabular data, retrieve best results from a table corpus, and perform question answering on structured data. The new tools include the TableTextRetriever for retrieval on a mixed corpus of documents and tables, and the TableReader for question answering on tables. These tools have been designed to work with retriever and reader models trained specifically on datasets with structured tabular data. The Haystack pipeline allows users to combine these nodes for table question answering on a large collection of tables. The tools are available for use in Haystack's GitHub repository, and the community is invited to join the Discord server for support and updates.