How ShareChat Scaled their ML Feature Store 1000X without Scaling the Database

ShareChat, a leading social media platform in India, faced significant challenges in scaling its machine learning feature store to meet the demands of its rapidly growing user base, particularly for its short-form video app, Moj. Initially, the system, based on ScyllaDB, struggled to handle the required scalability, becoming unresponsive at 1 million features per second when the goal was to achieve 1 billion. Through performance optimizations, the team restructured the database schema, improved cache locality, and implemented consistent hashing, which allowed them to significantly reduce the load on ScyllaDB while achieving their scalability goals without expanding the database infrastructure. By splitting the feature service into multiple deployments and leveraging technologies such as Envoy Proxy for better observability and caching, ShareChat improved the cache hit rate and managed to scale the feature store to handle 3 billion features per second. The process underscored the importance of using proven technologies, the incremental nature of optimizations, and the need for practical solutions, even if they are not the most elegant.