When users don’t click thumbs up: Inferring agent feedback from Datadog telemetry

Datadog researchers describe using weak labeling to estimate agent user satisfaction from existing telemetry when users rarely provide explicit thumbs-up or thumbs-down feedback. Their public session classification skill combines Agent Observability traces, Real User Monitoring data, and Audit Trail records to infer whether a chat interaction was positive or negative, using conversation sentiment, user behavior, and resulting changes to Datadog artifacts. Tested against hundreds of manually labeled Bits Chat sessions, the classifier achieved 78% accuracy with traces alone, 80% after adding RUM, and 82% with Audit Trail data, indicating that additional telemetry improves its usefulness. Although it is not intended to replace direct user feedback, the approach can help teams prioritize sessions for review, generate approximate labels for evaluation datasets, and monitor agent performance when explicit feedback is limited.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.