Realtor.com on guardrail metrics, the 30/30/30 rule, and AI as your junior data scientist

Whitney Perez, Director of Product Management at Realtor.com, discusses with GrowthBook CMO Ashley Stirrup how effective experimentation depends less on simply running A/B tests than on establishing reliable data, shared testing knowledge, clear guardrails, and peer review processes. Drawing on experience at Citrix, GoDaddy, and Realtor.com, Perez advocates the “30/30/30” expectation that roughly equal portions of experiments will win, lose, or be inconclusive, helping teams value learning rather than chase only positive results. She illustrates the importance of secondary and guardrail metrics through a Realtor.com bundling test that appeared to increase attach rate by 300% but ultimately reduced total revenue because an added checkout step caused customer drop-off. Perez argues that experimentation skills are accessible to non-specialists who understand core concepts such as hypotheses, decision metrics, traffic requirements, confidence intervals, and confounding variables, while organizations should connect team-level metrics to broader North Star goals and use qualitative research before testing. She sees AI as a useful tool for analyzing data and uncovering opportunities, but emphasizes that human review remains necessary because AI can produce confident but incorrect conclusions. Her priorities for Realtor.com are dependable instrumentation, experimentation knowledge distributed across teams, and a culture that welcomes positive, negative, and inconclusive outcomes.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.