Teaching AI to speak Splunk, then proving it works

The text discusses the challenges and solutions associated with migrating from Splunk to Axiom, focusing on the translation of Splunk’s Search Processing Language (SPL) to Axiom Processing Language (APL). Manual migrations are time-consuming and prone to errors, so the company developed AI-powered translation skills to automate the process, significantly reducing the time required for migration. These skills include "spl-to-apl" for translating queries and "building-dashboards" for creating dashboards. To ensure reliability, the company implemented evaluation (eval) tests to measure the accuracy of AI-driven translations, integrating these tests into their continuous integration/continuous deployment (CI/CD) pipeline via GitHub Actions. This approach replaces reliance on manual spot-checks with data-driven evaluations, allowing teams to detect and rectify errors before they reach production. The company also emphasizes the importance of documentation and repeatable testing, highlighting that their eval framework is available to all Axiom customers, facilitating structured, reliable AI development and deployment.