At a growing company like Intercom, maintaining high-quality conversations with customers is crucial for building valuable relationships and delivering world-class support. To achieve this, measuring conversation quality is essential, but it can be challenging due to the variance in conversation quality among support agents. The company developed a peer review system called Conversation Carousel to address this issue. This tool helps onboard new staff members, identifies those who need further guidance, and ensures that all team members review a certain number of teammates' conversations every week. The system scores conversations based on tone, quality, and adherence to internal processes, providing feedback in the form of ratings and comments. By implementing this peer review system, Intercom has seen significant results, including improved conversation quality and a culture of collective upskilling among support staff.