Evaluating political bias in LLMs

The analysis of Grok 4, an AI developed by Elon Musk's xAI, reveals a complex political bias landscape that challenges initial expectations of right-wing propaganda. Contrary to media reports, Grok 4, along with other leading AI models like GPT-4.1, Gemini 2.5 Pro, and Claude Opus 4, leans left of center, with Grok 4 displaying a notably bimodal distribution that swings between extreme left and right positions, making it the most contrarian among its peers. The dataset used for this analysis, featuring 2,500 political questions, showed that Grok is unexpectedly critical of Musk's companies, suggesting a possible overcorrection for perceived bias. All tested models demonstrated a general left-leaning tendency on economic issues, reflecting the training data's influence rather than a deliberate ideological drive. The study highlights the inherent challenges of achieving true political neutrality in AI models and underscores the importance of transparency regarding AI biases.