How ABANCA delivered a safer AI assistant to 2 million customers, cutting evaluation costs by 71% | Galtea Blog

ABANCA successfully deployed its AI banking assistant, SofIA, to over 2 million customers without any production incidents, significantly reducing evaluation costs and improving security measures. The deployment of SofIA, which handles queries in multiple languages, faced challenges related to manual validation processes, which were costly and limited in scope. To address these issues, ABANCA implemented Galtea's platform, which uses specification-driven test generation and calibrated LLM judges to evaluate SofIA's compliance with policy guidelines. This approach allowed ABANCA to automate the evaluation of SofIA's performance against adversarial attacks and policy compliance, leading to a 71% cost reduction per evaluation cycle and surfacing 23.6 times more vulnerabilities than manual methods. The improved evaluation process not only reduced costs but also enhanced regulatory compliance, providing a 10x return on investment and mitigating significant regulatory risks. The deployment now includes a permanent evaluation pipeline that ensures every release candidate is thoroughly tested before reaching customers, meeting the documentation requirements of the EU AI Act.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.