July 2019 Summaries

Sentry introduced Snuba, its primary storage and query service for event data, to mitigate risk while transitioning databases. The transition was complex due to the need to ensure data integrity and performance without downtime or compromising trust with users. To address this, Sentry used a Service Delegator, which executed queries against both existing backends and new Snuba backends, allowing for concurrent requests and logging results to Kafka for analysis. This enabled the identification of issues, such as race conditions, and informed improvements to Snuba's performance, including reducing latency and increasing similarity between results. After months of testing and iteration in production, Sentry successfully transitioned to Snuba without user impact, gaining operational experience and confidence in the new system before launch day.

Sentry recently introduced Snuba, its new primary storage and query service for event data, backed by open source column-oriented database ClickHouse. To transition from its existing storage systems to Snuba, Sentry followed a complex process that involved testing and validation, including the use of an intermediary called the Service Delegator. The Service Delegator allowed Sentry to execute queries against both the old and new backends concurrently, enabling the collection of data on similarity between the two systems. This data was used to monitor performance and identify areas for improvement, ultimately leading to a successful launch with minimal user impact. By running Snuba in production for months before launching it to end-users, Sentry gained operational experience and confidence in the new system, making the launch day practically uneventful.

Sentry Meetups are launching a new event series called Community show-and-tell night, which will take place on July 31st, where developers can share their best fails and what they've learned from them. The format is informal, fun, and low-key, encouraging diverse topics and experiences. Liz Krane, the new Developer Advocate at Sentry, aims to build upon the community by expanding meetup repertoire and inviting every developer to participate in this casual environment. The goal is to empower developers to share their stories and make their voices heard. This event series is inspired by Learn Teach Code's successful show-and-tell lightning talk events, which featured diverse groups of first-time speakers sharing projects on various topics.

Sentry is an open-source company that believes in contributing to the developer community both online and offline. The company launched its Sentry Scouts meetups in January 2018, which featured panel discussions on various tech topics. As a new Developer Advocate at Sentry, Liz Krane aims to expand the meetup series with a new event format called Community Show-and-Tell Night. This event will feature lightning round show-and-tells where developers can share their best fails and what they've learned from them in an informal 5-minute story. The goal is to encourage every developer to share their experiences, explore diverse topics, and invite those new to public speaking to make their voices heard in a low-key and judgment-free environment.

Sentry's debug experience relies on source maps, which are used to map tokens in minified JavaScript files back to their original non-minified file names and line numbers. However, source maps have shortcomings that cause problems for platforms like Sentry, such as not fully equipping users with the information needed to understand and debug errors. Sentry tries to guess function names by tokenizing backward over a minified JavaScript file token by token but this approach can be flawed due to issues like local utility functions and ES6 method syntax. The current implementation of source maps poses challenges for safe fetching, processing, and providing customer-supplied source maps. To improve the ecosystem, collaboration between authors of minifiers and browser/debugger vendors is needed to evolve the source map format to better support scopes, WASM, and non-JavaScript languages.

The Sentry platform has limitations with source maps due to their shortcomings in providing accurate information about function names, which are crucial for debugging transpiled code. Source maps can map a token in a minified JavaScript file to its non-minified counterpart, but they do not provide enough information to understand and debug errors. Sentry's approach to handling source maps involves tokenizing backward over a minified JavaScript file to guess function names, which is flawed due to the unreliability of function names. The platform currently uses a hacky approach to recover function names, which can lead to incorrect results. To improve this, Sentry welcomes an improved debug standard that could be augmented to support new technologies like WASM and non-JavaScript languages.

Sentry for Data has optimized Airflow by introducing a new plugin called sentry-airflow, which collects, handles, and reports errors that occur within Airflow tasks upon execution, providing valuable context and insights into data pipeline failures. The plugin is designed to be easy to integrate, requiring only the SENTRY_DSN specification and minimal setup steps. With sentry-airflow, Airflow errors are collected by Sentry, allowing for quick identification of failed data pipelines and providing breadcrumbs for additional insight into previous tasks that might have also failed.

Sentry is a crucial tool for data teams, enabling high-quality leads and tailored marketing campaigns, as well as informing business decisions such as Sentry pricing and feature roadmap. Apache Airflow is used for executing data pipelines, but its built-in log interface lacks context, making issue resolution challenging. A new plugin, sentry-airflow, has been introduced to address this issue by collecting, handling, and reporting errors that occur within Airflow tasks. The plugin can be easily integrated into Airflow via a drop-in solution requiring only two steps: placing it in the plugins folder and setting an environment variable. Errors are reported with useful tags, including dag_id, task_id, execution_date, and operator, and also include upstream DAG tasks as breadcrumbs for additional insight. The plugin is available for use and continuously improving, with support engineers available to help with feedback, feature requests, or usage assistance.

In this post from Sentry, we explore how to resolve issues with ease as part of their workflow series, which aims to optimize the development process from crash to resolution. The focus is on creating a culture and infrastructure equipped to handle unexpected issues, providing visibility into code to enable a holistic picture of the service. The post discusses using commit integrations and features with source code management providers to assign issues to owners, referencing issues in commits, and marking issues as resolved in future releases. This enables developers to focus on developing and fixing issues without tracking them manually. Deploy emails can also be configured to provide details on changes made to production. By implementing these steps, the development workflow becomes seamless, making bug fixing a fluid part of the process.

The Sentry Workflow is a series of articles that aims to optimize the development process from crash to resolution, focusing on minimizing impact and creating visibility into code. The workflow has reached its final stage by resolving issues with ease, allowing developers to stay organized and productive. This involves referencing issues in commits, marking them as resolved, and configuring deploy emails to send details on deployed changes. By doing so, errors become a fluid part of the development process, reducing time spent investigating and testing.