Why observability tools are missing critical debugging data (no matter how you sample)

In the realm of observability, collecting more data does not necessarily equate to better debugging capabilities, as traditional tools are adept at monitoring distributed systems but fall short in capturing comprehensive debugging information. These tools gather traces, logs, and metrics, offering visibility into occurrences and locations of issues, but they fail to capture crucial data like request/response payloads, details of external API calls, and frontend context. Unsampled observability does not resolve these gaps, as it merely provides more metadata without the necessary payload details. To address this, some teams resort to custom logging, which is labor-intensive and costly, while others face challenges in correlating frontend and backend data due to manual processes. A more efficient approach is session-based recording, as exemplified by tools like Multiplayer, which allows for on-demand collection of detailed, correlated data for specific sessions, providing a complete picture that is more cost-effective and useful for both human and AI-assisted debugging. This method emphasizes capturing the right data automatically and ensuring it is accessible and correlated, thus enhancing the effectiveness of AI tools and reducing the burden on engineering teams.