Verification Is Not Testing. Understanding the Difference Could Save Your Codebase

In the context of AI-generated code, testing and verification serve distinct roles, with testing focused on ensuring code behaves correctly and verification ensuring it aligns with pre-established agreements. Traditionally, humans have carried the intent behind code, allowing for informal verification through code reviews. However, AI agents lack intent and rely solely on prompts, which can lead to code that passes tests but does not meet agreed-upon specifications. Aviator's experiment demonstrated that AI can efficiently verify code against a detailed specification, identifying intent mismatches that traditional CI pipelines might miss. This highlights the need for explicit, human-approved specifications and verification as distinct stages in the development process, with testing continuing to address bugs and regressions. Verification has thus become a critical layer in the pipeline, ensuring that AI-generated code aligns with human intentions, while reducing the reliance on human code reviews for extensive, line-by-line checks.