Moving Beyond EdgeRank for Personalized Newsfeeds

The blog post, divided into two parts, explores the development of personalized news feeds by initially examining a basic EdgeRank-inspired algorithm and then advancing to more sophisticated machine learning techniques. In the first part, the author discusses the theoretical underpinnings of their Instagram discovery engine, focusing on three components: affinity, weight, and time decay, which are combined to rank posts. Affinity is measured using the personal PageRank algorithm, weight is influenced by likes and comments, and time decay ensures newer content is prioritized. In the second part, the author highlights the limitations of simple algorithms for complex data and introduces boosted decision trees and neural networks as techniques to enhance feed ranking. These machine learning models can handle numerous parameters and optimize engagement by leveraging contextual and historical features, such as user interactions and content characteristics. The post emphasizes the potential of combining boosted decision trees with logistic regression and neural networks to improve efficiency and performance, while also considering transfer learning to mitigate training costs.