Santosh Vempala, a professor at Georgia Tech, discusses the phenomenon of hallucinations in language models during a talk centered around the OpenAI paper "Why Language Models Hallucinate," which he co-authored. The paper provides a theoretical framework linking hallucinations to classical misclassification theory and statistical calibration, explaining that pre-training encourages models to produce false-yet-plausible statements by maximizing data likelihood rather than accuracy. Vempala highlights that hallucinations can be mathematically predicted, with the hallucination rate being directly tied to the misclassification rate. He emphasizes that post-training should aim to penalize "confidently wrong" answers, encouraging models to abstain from guessing when uncertain, thus fostering behavioral calibration. This approach involves rewarding uncertainty and penalizing incorrect responses to reduce hallucinations over time. Vempala also clarifies that hallucinations are a statistical issue rather than one dependent on specific model architectures and that creative outputs generated by models can be contextually understood as hallucinations when they extend beyond factual data.