AI in Production Is Growing Faster Than We Can Trust it

Enterprise software has evolved from testing generative AI models to integrating them into product interfaces and infrastructure, driven by the need for competitive advantage. This shift, similar to the adoption of microservices, involves transforming the technology stack to accommodate the varied inputs and outputs of large language models (LLMs), which increases the risk of system failures due to their probabilistic nature. AI reliability and observability have become critical concerns, with predictions that by 2028, 40% of organizations will implement AI observability to manage new failure patterns such as hallucinations, model drift, and latency issues. Traditional observability platforms struggle with the complexity of AI telemetry, prompting the development of specialized tools that can track high-dimensional data for real-time insights. Companies like Intercom have successfully leveraged these tools to optimize performance and costs, as seen with their AI agent Fin.ai, which improved response times and scalability through detailed monitoring and data analysis. To ensure safe scaling of generative AI features, platforms must offer unlimited cardinality and high dimensionality, enabling engineers to discern meaningful signals from the noise of high-volume AI workloads.