From bottleneck to self serve: scaling experimentation at US Bank

Vijay Lal, Lead Product Manager for Experimentation at US Bank, describes how regulated organizations can scale customer testing while maintaining security, reliability, and rigorous oversight. Facing demand that exceeded the capacity of a central experimentation team, US Bank adopted a self-service platform supported by simplified tools, continuous training, user accountability, and guardrail metrics. A redesign of the bank’s secure login widget illustrated this approach: after testing revealed that slower connections could prevent some customers from loading the embedded widget, the team introduced a fallback experience to ensure no users were excluded and experiment coverage remained complete. Lal also emphasizes that experiments should begin with a clear hypothesis, which then determines primary, secondary, and guardrail metrics, rather than selecting KPIs after a decision has already been made. The discussion argues that effective experimentation requires organizations to democratize access responsibly, account for edge cases and performance differences, and measure behavior with metrics closely connected to the feature being tested.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.