Pytest for Agent-Generated Code: Concrete Testing Strategies to Put Into Practice

Agents writing tests must prioritize intent alongside correctness, with clear testing levels, preferred patterns, and explicit anti-patterns providing the necessary structure for creating rapid, reliable Pytest suites that scale with automation. Explicit guidelines ensure that agents maintain high-quality standards, avoiding subtle errors or long-term maintenance issues, while repository-wide strategies dictate what constitutes a "good test," when tests should be conducted, and what types of tests are suitable for various scenarios. Test sub-agents work within a constrained context to prevent the common failure of fixing tests by altering code under test, focusing instead on specific tasks like adding coverage, reproducing bugs, or refactoring. The strategy encourages Test Driven Development for new features and outlines distinct workflows for bug fixes, emphasizing the importance of writing appropriate tests for new code changes. Clear anti-patterns are established to prevent performance erosion and ensure that tests remain lightweight and resilient, with rules to avoid unnecessary subprocesses or filesystem interactions. By defining testing levels and aligning individual test rules with broader objectives, the strategy ensures a consistent, sustainable approach as the codebase expands, highlighting the foundational role of a well-defined testing strategy in maintaining correctness, confidence, and long-term maintainability in automated development.