Burn less, ship more: the case for token optimization

During a period dubbed "tokenmaxxing," parts of the tech industry, including companies like Meta and Microsoft, attempted to measure AI adoption by tracking token consumption, leading to wasteful practices and inflated costs. This approach, driven by Goodhart's Law, where a measure becomes ineffective when it turns into a target, led to engineers deliberately wasting tokens to avoid being seen as insufficiently AI-native. Despite the high token consumption, no significant improvement in software quality was noted, prompting a shift towards token optimization. This transition is inevitable due to fiscal pressures, as AI providers' pricing does not reflect the true cost of inference, and environmental concerns, with AI's energy footprint becoming a growing issue. Token optimization involves providing AI agents with precise data, selecting appropriate models for different tasks, practicing context minimalism, and teaching AI to be less verbose, ultimately resulting in lower costs and higher-quality software.