Serving article comments using reinforcement learning of a neural net

Yahoo's implementation of an advanced comment serving system on its content sites like Yahoo Finance, News, and Sports uses reinforcement learning and the Vespa open-source platform to efficiently manage and rank large volumes of user comments. By storing comments as individual documents in Vespa, the system can handle real-time operations such as adding, finding, and displaying comments, even when facing millions of comments with varying degrees of relevance and interest. The comment ranking process involves computing a score for each comment using a ranking function configured in Vespa, which incorporates features based on user interactions and author reputation. The team improved comment selection by transitioning from a manually written ranking expression to a neural network model that leverages reinforcement learning to optimize user interest proxies like dwell time and user engagement signals. This approach has resulted in a 20% increase in average dwell time, with the neural network evaluated in parallel across content nodes to maintain low response times. Future plans include expanding this methodology to other domains such as personalized content recommendations.