Keller Williams Realty, the largest real estate franchise by agent count, is advancing its AI and agent engineering initiatives, emphasizing the importance of establishing robust frameworks and evaluation practices from the outset, according to its senior analytics engineer, Koby Close, and software architect, Venkat Chinni. The company's efforts in integrating generative AI (GenAI) focus on improving the efficiency of its CRM tools and empowering agents, with initial use cases targeting internal efficiency within the tech department. Their partnership with Arize on the MLOps platform has facilitated tackling technical challenges, such as the development of a text-to-SQL product, which revealed the necessity of detailed documentation and a semantic layer to enhance AI performance. The role of the agent engineer at Keller Williams combines AI capabilities with business value, requiring a blend of software engineering skills and an understanding of model behavior, emphasizing prompt design, orchestration, and collaboration with product and compliance teams. Through the use of the Arize platform, Keller Williams has been able to identify inefficiencies in AI tool deployment, which has led to more efficient operations and cost savings.