ML-Driven Test Intelligence at Scale [Testμ 2026]

Tanvi Mittal’s Testμ Conf 2026 session describes test intelligence as a decision-making layer focused on choosing what to test, defining meaningful assertions, and evaluating the trustworthiness of automated evaluators rather than merely generating more AI-produced tests. She argues that AI systems challenge traditional testing assumptions because their outputs, oracles, contexts, and change surfaces can vary, making capability different from reliable performance. Her proposed closed-loop approach combines known risks and production signals through observation, prioritisation, context reconstruction, assertion design, validation, decision-making, and feedback into evaluation suites, with candidates assessed using inspectable factors such as novelty, exposure, impact, and reproducibility. Mittal emphasizes privacy-first handling of production data through redaction before analysis, explains that anomalies are test candidates rather than automatically defects, and stresses that generated tests require meaningful invariants to demonstrate real coverage. She also cautions that LLM-based judges should be evaluated against human-labelled cases and analyzed across important slices before being used for release decisions. LogMiner QA, an open-source tool she has worked on, illustrates part of this approach by sanitising production logs, identifying unusual behavior, prioritising relevant patterns, and producing candidate tests. Her practical recommendation is to begin with one important AI-assisted workflow, instrument its lineage, and convert one sanitised, production-derived behavior into a validated regression test while retaining human oversight for uncertain decisions.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.