January 2026 Summaries

The text discusses the challenges and solutions associated with migrating from Splunk to Axiom, focusing on the translation of Splunk’s Search Processing Language (SPL) to Axiom Processing Language (APL). Manual migrations are time-consuming and prone to errors, so the company developed AI-powered translation skills to automate the process, significantly reducing the time required for migration. These skills include "spl-to-apl" for translating queries and "building-dashboards" for creating dashboards. To ensure reliability, the company implemented evaluation (eval) tests to measure the accuracy of AI-driven translations, integrating these tests into their continuous integration/continuous deployment (CI/CD) pipeline via GitHub Actions. This approach replaces reliance on manual spot-checks with data-driven evaluations, allowing teams to detect and rectify errors before they reach production. The company also emphasizes the importance of documentation and repeatable testing, highlighting that their eval framework is available to all Axiom customers, facilitating structured, reliable AI development and deployment.

Axiom introduces a comprehensive user feedback capture system designed to enhance AI capabilities by linking user feedback directly to AI traces, thus allowing teams to identify and address issues more effectively. This system collects both explicit feedback, such as thumbs up/down and comments, and implicit signals like regenerations or copies, integrating them into a dedicated view in Axiom's Console for analysis. By associating feedback with specific AI traces, domain experts can prioritize and examine problematic interactions, transforming them into test cases to prevent future occurrences. This feedback-driven approach not only aids in immediate issue triage but also informs the development of a robust evaluation suite based on real-world edge cases, fostering a continuous improvement loop. Axiom's strategy combines this feedback mechanism with existing observability and offline evaluations, creating a holistic system that bridges the gap between user experience and quality assurance, ultimately leading to more reliable AI deployments.