Introducing DigitalOcean Signals and Simulations: see how your agents and models really behave in production, then test the fix before it ships

DigitalOcean has introduced Signals and Simulations in public preview as part of its Agent Learning Services to help teams identify and test fixes for agent failures that conventional observability may miss. Signals monitors production activity from Managed Agents and the Inference Engine, grouping interactions into sessions and flagging issues across user sentiment, tool execution, and infrastructure conditions, while using low-cost triage to prioritize informative traces rather than grading every interaction. Flagged production sessions can be converted into evaluation scenarios, and Simulations uses these or user-defined scenarios to test agent or model changes through repeated, multi-turn conversations in an isolated environment with production-like configurations and tools. An LLM judge evaluates trajectories on metrics including goal completion, knowledge retention, and tool-use accuracy, allowing teams to compare baselines and candidates and distinguish reasoning problems from tool failures. The capabilities are designed to keep customer traces within a DigitalOcean team boundary, offer PII redaction, support mocked tools for actions involving data changes or costs, and enable saved scenarios to become regression tests, although preview support excludes some third-party agent frameworks.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.