May 2026 Summaries

Experiment Approvals, a new feature from LaunchDarkly now available in beta, provides a governance framework for managing experiments in production environments, particularly benefiting teams in regulated industries or those with strict change management practices. This feature extends LaunchDarkly's existing approval workflows for feature flags to encompass experimentation, ensuring that every experiment, whether starting, modifying, stopping, or shipping, undergoes a structured review process. By requiring approvals for key experimental changes, the system enhances oversight and accountability without hindering the agility of product teams. It includes centralized interfaces for reviewing and approving requests, integrates with tools like Slack and email for notifications, and maintains audit logs for transparency and compliance. Furthermore, Experiment Approvals is designed to facilitate safe experimentation in agent-driven future scenarios, allowing organizations to define which actions require human intervention.

Agent Optimization, now available in beta for eligible customers within the AgentControl platform, automates the improvement of agents by generating and testing combinations of models, prompts, and hyperparameters against predefined acceptance criteria. This method expands beyond the limitations of manual iteration, as it explores configurations not previously considered, thus potentially finding optimal solutions that are more efficient and cost-effective. Two modes of optimization are offered: Exploratory mode for mapping behavior across diverse inputs and Expected Output mode for refining performance without losing established functionality. Once a successful configuration is identified, it can be seamlessly promoted to production, allowing for gradual implementation and performance tracking through AI Insights. The ultimate goal of AgentControl is to enable agents to autonomously identify and apply improvements based on real-world performance, minimizing the need for manual intervention.

AgentControl is a new operational platform designed to manage the challenges of running AI agents at scale, addressing the unpredictability and degradation issues that arise after deployment. Traditional software remains stable after deployment, but AI agents are inherently dynamic, leading to potential performance declines without direct code changes. This platform integrates with LaunchDarkly's flag delivery infrastructure to allow rapid updates to agent behavior elements such as prompts and models, offering centralized governance and control across organizations. AgentControl includes features like offline evaluations, guarded rollouts, online evaluations, AI insights, and experimentation capabilities to ensure agents meet quality, cost, and reliability standards before issues impact users. It provides real-time observability and connects it with actionable controls, enabling teams to preemptively address potential problems efficiently. The platform is designed to evolve continuously by feeding production data back into configurations, thus enabling automatic system improvements without manual interventions.

Adaptive Triggers, now in closed beta, addresses the challenge of minimizing the time between detecting an issue in a production AI system and implementing a fix by automating the response process. Unlike traditional software, AI systems are unpredictable due to factors like model updates and environment changes, which are often outside a team's control. With Adaptive Triggers, teams can define rules on configurations so that when a monitored metric breaches a threshold, the system automatically switches to a predefined backup variation without human intervention. This seamless switch, facilitated by AgentControl within the application, occurs in under 200 milliseconds, ensuring uninterrupted user experience. The feature extends to various signals in the observe-and-act loop, allowing for automatic adjustments in response to quality score drops or cost spikes and restoring primary variations when metrics recover. To access Adaptive Triggers, interested teams are encouraged to contact their account managers.

LaunchDarkly has introduced AgentControl, a new solution designed to manage AI agents in production, expanding beyond its decade-long focus on controlling code releases. This tool allows teams to configure, evaluate, observe, and control agents without needing separate tools, reflecting a shift in software development in the AI era. AgentControl offers real-time control through features like runtime-changeable AI configurations, allowing teams to experiment and validate changes offline, then monitor live traffic to manage latency, costs, and behavioral drift. It supports automated actions such as rerouting traffic or rolling back changes to mitigate risks before they affect customers, using the same infrastructure that supports extensive daily evaluations for numerous companies. The launch of AgentControl aims to help teams govern agents, optimize AI performance, and continuously improve in production, emphasizing the need for runtime control in the fast-evolving AI landscape.

LLM observability is a comprehensive approach to monitoring and improving the behavior of large language models (LLMs) in real-world applications, focusing on both technical and semantic performance metrics. Unlike traditional systems, where performance is often measured by uptime or latency, LLM observability emphasizes understanding the probabilistic nature of model outputs, which are influenced by hidden reasoning and stochastic sampling. This process involves tracking inputs, outputs, latency, and other key metrics to detect anomalies, refine prompts, and maintain reliability, thus building trust and accountability. Effective observability encompasses multiple layers, including data and prompt monitoring, model performance evaluation, cost and error tracking, user experience and risk management, and controlled rollouts. It transforms LLMs from opaque systems into transparent, measurable, and improvable frameworks, ensuring alignment with business objectives and ethical standards. LaunchDarkly's AI Configs, through feature flags and versioned configurations, enhance observability by enabling real-time experimentation and progressive delivery, allowing teams to manage, analyze, and improve LLM performance dynamically while minimizing the risks of deployment.