Company
Date Published
Author
MongoDB
Word count
1589
Language
English
Hacker News points
None

Summary

The author, a MongoDB engineer, created a program called "followt" to track Twitter followers using MongoDB. The program retrieves the follower lists of accounts from Twitter's REST API and updates the corresponding documents in MongoDB with the new follower information. The author designed a simple schema to store the data, which proved more efficient than a complex relational design. The program uses set intersection algorithms to find users who have unfollowed or followed, using both naive multisearch and merge approaches, as well as a "mark-and-sweep" approach that exploits the timestamps in the documents. The author tested the performance of these algorithms on follower lists of different sizes and found that the "mark-and-sweep" approach was the fastest. The program was run for several days to track the development of the @MongoDB account and other accounts, showing a high follower growth rate and average loss rate, which adds an interesting twist to the current database popularity ranking.