July 2020 Summaries

The Sentry Release GitHub Action is a tool that automates release management for developers by integrating with GitHub Actions. It simplifies the process of deploying code to an environment and notifies Sentry of new releases, surfaces suspect commits in case of errors, and views un-minified JavaScript source maps. The action can be easily installed using a YAML file and requires setting up environment variables such as SENTRY_AUTH_TOKEN, SENTRY_ORG, and SENTRY_PROJECT. Once set up, the action automates the deployment process, allowing developers to focus on resolving errors quickly and efficiently.

Lazy-loading is used by Sentry to improve the efficiency of its User Interface by reducing the time it takes for data to load. Instead of fetching all data at once, Sentry splits the request into two parallel requests, showing the results immediately after the first request finishes. This approach allows users to see some data almost instantly while waiting for other data to render. By using this technique, Sentry was able to achieve a 22% faster UI and save half a second in load time. The goal is not only to provide error monitoring but also to ensure that customers can see the data they need in a timely manner.

API Authentication Bypass The Sentry API authentication bypass vulnerability was discovered on July 20th, 2020, affecting customers who had custom integrations with the platform. A patch was deployed shortly after and an investigation revealed no evidence of customer data exposure or exploitation in the wild. The challenge in forensics came from the lack of association between authentication tokens and requests, making it difficult to identify valid vs. invalid requests. To overcome this, Sentry used Clickhouse, a database that allows for quick filtering and correlation of log data. By narrowing down the scope using techniques such as clustering data by organization and service provider, Sentry was able to reduce the list of potentially affected customers from nearly 4,000 to 462. The investigation also highlighted the need for improvements in metadata management, penetration testing, customer communication, forensic tooling, and automated testing.

GraphQL errors can be attributed to one specific phase of the request lifecycle, distinguishing between validation and execution errors, which have different implications for client-side tooling. Validation errors are typically user-facing, while some execution errors may contain only limited information by design. When reporting errors, GraphQL helps to leverage the strict API schema to reconstruct a failing case relatively quickly based on request and response details. An error handling implementation using Apollo Server and Node.js connected to Sentry can be used to report errors with additional context, including operation type, query, variables, and user input. This allows for filtering and inspection of reported events in Sentry, enabling better investigation and troubleshooting of client-side opaque request failures.

Performance is Sentry's new code performance monitoring offering that provides deeper visibility into API calls and database queries, helping developers deliver fast customer experiences with just five lines of code. It offers features such as User Misery, which pairs industry metrics with user metrics to identify slow APIs and improve troubleshooting, Mission-critical Context, which surfaces transactions by duration time, related code errors, and impact on the customer, and Untangle Tracing, which traces frontend back to its API calls and slow database queries. The tool helps developers resolve user misery by getting out in front of painful experiences before they become problems and problems before they cause customers to churn. It is designed to help developers deliver performant and reliable software and has been used successfully by companies such as Tackle and SmugMug.

A recent widespread crash issue was caused by a change in the Facebook SDK, which attempted to hit an API endpoint but crashed due to an invalid callback function and failure to check for null values. The problem was resolved on the server-side, but highlights the risks of relying on third-party SDKs, which can pose a single point of failure with significant downstream consequences, including spikes in support cases and issues with error monitoring tools. To prevent similar crashes in the future, developers should consider implementing defensive programming techniques, such as validating data, using schema validation libraries, and retrying operations carefully to avoid uncontrolled loops.

Sentry's new Metric Alerts feature allows developers to monitor the overall health of their applications by setting alerts on the frequency of any subset of events. This helps in identifying potential issues and root-causing problems quickly. Users can filter on event tags or attributes, set separate warning and critical thresholds, send notifications to email and Slack, view active and resolved alerts, and jump from an alert into a curated Discover view for quick analysis. Metric Alerts also provide visibility into error patterns, helping teams eliminate redundant issues. The feature is expected to be extended in the near future with additional capabilities.

Sentry Data Wash Now Offering Advanced Scrubbing` Sentry, an error monitoring platform, has introduced advanced data scrubbing features for its users. This addition aims to provide greater control and more tools to help users choose which data to redact from events. The new feature builds upon Sentry's existing server-side data scrubbing capabilities, allowing users to define custom regular expressions, hash sensitive data, and limit the scope of individual "rules" to specific parts of an event. This updated functionality is exposed through a new rule-based system, enabling users to configure their scrubbing settings in addition to existing features. The feature is now available to all users, with previously processed events still requiring manual deletion to remove sensitive data. Sentry continues to develop more features to complement the advanced scrubbing capabilities, including applying scrubbing settings before sending data to its servers and improving rule iteration.