Building production AI for regulated industries with a leading digital insurer

A leading digital insurer partnered with Baseten to develop a state-of-the-art AI system that efficiently handles customer insurance queries while maintaining strict compliance with Financial Conduct Authority (FCA) regulations and delivering sub-3-second response times. The project involved deploying a fine-tuned open-source model that surpassed existing closed-source models, including GPT-5, in reasoning capabilities and speed. The development process was divided into several phases, starting with building a sophisticated knowledge retrieval system and evolving through a comprehensive evaluation framework that identified failure modes and refined prompts for consistency and accuracy. Genetic optimization and supervised fine-tuning were employed to create high-quality training data, with reinforcement learning further enhancing the model’s ability to perform precise retrievals and generate compliant responses. By eliminating vector embeddings in favor of grep-based text search, the system improved robustness and adaptability to document changes. After six weeks, the AI achieved a 95.7% compliance pass rate, significantly outpacing competitors in speed and accuracy, and set the foundation for scaling the system to support additional insurance products and more complex interactions.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.