August 2022 Summaries

The Learning to Retrieve Passages without Supervision paper by Ori Ram et al. explores Self-Supervised Learning as an alternative route to training new models without human labeling. They introduce Span-based Unsupervised Dense Retriever (Spider), a recent breakthrough for Self-Supervised representation learning applied to text retrieval in search. Spider's results show that it achieves similar performance to Supervised models on unseen data distributions, demonstrating strong Zero-Shot Generalization capabilities. The authors further illustrate how we can target Spider's performance to a particular data distribution with Transfer Learning, requiring only 128 labeled examples. This research opens up new possibilities for training custom retrieval models without the need for large labeled datasets.

The new GOMEMLIMIT feature in Go 1.19 helps prevent out-of-memory (OOM) situations by allowing developers to set a soft memory cap that makes the Garbage Collector more aggressive when memory is scarce, ensuring efficient use of available memory and preventing unnecessary OOM kills. This feature is particularly useful for applications with long-lived memory requirements, such as databases or caches. By setting an appropriate GOMEMLIMIT value, developers can ensure their applications run smoothly without running out of memory.

Semantic search overcomes limitations of keyword-based search by using machine learning models like Bi-Encoder and Cross-Encoder in a vector database. Bi-Encoders are fast but less accurate, while Cross-Encoders are more accurate but slower. Combining these two models can improve the search experience by first using Bi-Encoders to retrieve a list of result candidates and then using Cross-Encoders for reranking the most relevant results. This approach benefits from both efficient retrieval and high accuracy, making it suitable for large scale datasets.