January 2025 Summaries

Technical: One Right Answer or Many? A Useful Distinction for Evaluating and Fine-Tuning LLMs` This article explores the distinction between deterministic tasks, which have one correct output for a given input, and freeform tasks, which have many potential right answers. The author highlights that this distinction is crucial when evaluating and fine-tuning Large Language Models (LLMs). For deterministic tasks, techniques such as using temperature=0, golden datasets, and smaller models can be effective. In contrast, freeform tasks require methods like vibe checks, LLM-as-judge, user feedback, or preference-based approaches like Direct Preference Optimization (DPO) or Reward Learning from Human Feedback (RLHF). By understanding this distinction, developers can tailor their approach to the specific task at hand and optimize their LLMs for better performance.