LMArena is a cancer on AI

LMArena, a popular online leaderboard in the AI community, is criticized for prioritizing engagement metrics over factual accuracy, leading to a flawed evaluation system where superficial attributes like verbosity, formatting, and emotive elements are rewarded over correctness. This system, open to the public and reliant on unpaid volunteers, lacks quality control and encourages behaviors that exploit human attention spans rather than promote rigorous assessment. The critique highlights several instances where incorrect responses were favored due to their presentation, illustrating a broader issue of misalignment between the leaderboard's metrics and the desired attributes of AI models, such as truthfulness and reliability. The text argues for a fundamental reevaluation of the values and practices guiding AI development, urging leaders to prioritize substantive quality over the allure of leaderboard rankings, as some frontier labs have successfully done by adhering to principled development strategies.