Unsexy AI Failures: The PDF That Broke ChatGPT

AI models like ChatGPT and Google's Gemini often face challenges when applied to real-world tasks, as demonstrated by Josephina, a teacher, who struggled to extract clean text from a PDF using ChatGPT. Despite the models' capabilities to excel in academic benchmarks, they frequently falter in practical applications such as parsing documents, recognizing ambiguity, and following instructions consistently. Josephina's attempts with both ChatGPT and Gemini revealed shortcomings like misinterpreted instructions, garbled outputs, and hallucinated responses, highlighting the gap between AI's theoretical prowess and its practical utility. These issues underscore the need for AI systems to address everyday tasks effectively, as their failure can significantly impact user trust and reliability, despite their impressive performance in controlled environments or demos.