September 2023 Summaries

I worked on enabling product developers to run queries against production data, allowing them to debug and validate their queries without deploying their feature. This project was heavily influenced by the company's excellent dogfooding culture and required collaboration with various teams, including security and SREs, to ensure compliance and authorization. I also contributed to another project, an asynchronous queries tool that allowed for running longer-running queries asynchronously in a worker process, teaching me about system design and scalability. Throughout my internship, I was grateful for the experience, acquiring new technical skills, working with a team of competent engineers, and meeting various people across the organization, who made this experience truly unforgettable.

Gino, a student at the Vienna University of Technology, spent his summer 2023 as an intern with Sentry's Mobile SDK team, where he worked on developing the Kotlin Multiplatform SDK, a cross-platform framework that allows developers to share code between their apps with Kotlin. This project was challenging due to the technology's instability and imperfect API and developer experience, but also presented opportunities for growth and innovation. Gino chose Sentry as his internship destination because of its innovative tech culture and meaningful contribution opportunities in Vienna. During his internship, he received a great amount of support from his team and enjoyed various activities and events hosted by Sentry, which valued all employees' comfort and well-being.

My main internship project was to detect, disable and notify users of broken integrations with Sentry's integrations for Slack, GitHub, and other platforms. I created a database using Redis to record integration request responses and used it to detect, disable, and notify users of broken integrations. I worked on improving the health of Sentry integrations and implemented a new method for looking up Slack channels when creating alert rules, which dramatically accelerated setting up Slack channel alert rules. Throughout my internship, I was treated like any other software engineer, participated in team brainstorming, and had real responsibility and impact on users. The community was extremely supportive and excited about my learning, with engineers caring about my projects and celebrating my progress. I learned a lot, had fun, and became a stronger engineer at Sentry, and the experience was instrumental in preparing me for future endeavors.

Michelle Fu, a student at Stanford University, spent her summer as an intern with Sentry Enterprise team where she worked on integrating Sentry with Opsgenie, an alert and on-call service owned by Atlassian. Her task was to build a new integration that could be configured at the organization level and send both issue and metric alerts. Michelle faced technical and design challenges due to Opsgenie's quirks but received positive feedback from users and fellow engineers for her work. She also had an enjoyable experience working with Sentry, learning from her mentor and teammates, and participating in the company's annual Hackweek.

I worked on analyzing customer stories and creating structured guidelines for writing future customer stories, as well as assisting in the launch of a new feature for Session Replay related to detecting rage and dead clicks. I also created a specialized landing page that served as a hub for content relating to specific product areas. Through these projects, I gained concrete skillsets related to product marketing that I will use in the future. I was grateful for the guidance from all the people I met during my time at Sentry and enjoyed the welcoming atmosphere and supportive culture of the company. The internship provided a well-rounded experience with autonomy and consistent support, allowing me to take ownership of projects and venture into creative solutions.

Spencer, a student at the University of Toronto, spent his summer 2023 as a software engineer intern with the Sentry Issues team, where he worked on integrating Sentry Issues with Discord. He chose Sentry for his internship after being impressed by the company and its product during the interview process. During his time at Sentry, Spencer enjoyed learning from various team members and adapting to office life, which was a significant adjustment from his previous fully remote work experience. The interns were assigned to different teams and received support from their mentors, allowing them to make meaningful contributions to the project while also gaining valuable product knowledge.

I worked on two impactful projects during my summer 2023 internship with Sentry, a DevOps platform that helps developers monitor and manage their software applications. I first contributed to the development of a deployment visualizer, a React-based application that provides a user interface for Sentry's deployment pipelines, allowing engineers to quickly diagnose deployment errors and understand the current deployment status. I also extended the functionality of a Kubernetes agent that integrates cluster error tracking with Sentry Crons, improving the reporting of Kubernetes cronjob events and integrating them with the new feature, driven by feedback from Sentry users who wanted a Kubernetes solution for their recurring jobs. Throughout the internship, I gained valuable experience working with wonderful mentors, learned to tailor projects to maximize customer satisfaction, and developed diverse technical skills addressing the needs at Sentry in different ways.

This was Lucas Hatae's internship experience with the Sentry BizOps team, where he worked on improving customer retention and recruiting through dashboards, trackers, and documentation, gaining skills in data analysis and tools like Looker, Google Sheets, and Notion. He was grateful for the welcoming environment and supportive team, which made him feel like part of the team from day one. Lucas enjoyed the company culture and activities, including offsites and Hackweek, and appreciated being given real projects that would be implemented into the company, making his internship experience invaluable.

Nikki Kapadia, a student at Toronto Metropolitan University, spent her summer as an intern with Sentry's Performance team, where she worked on identifying performance trends and contributing to the development of the Performance Change Explorer feature. She was drawn to Sentry because of its innovative technology and global impact, having previously used the product in her previous role and being intrigued by its functionality. During her internship, Nikki enjoyed the autonomy to set her own milestones and work towards an end goal, as well as the support and guidance from her team and intern coordinators. Throughout her experience, she found the company culture to be welcoming and collaborative, with opportunities for networking and skill-building. The internship program allowed her to work on a impactful project, learn from her team members, and expand her technical knowledge. Overall, Nikki's internship at Sentry was a unique and enjoyable experience that enhanced her skills and knowledge in software development.

Flutter is a powerful toolkit for building beautiful, natively compiled applications for mobile, web, and desktop from a single codebase. Debugging and profiling are crucial aspects of developing any application, and Flutter provides an extensive suite of tools to help developers identify and fix issues in their apps. This includes the default Flutter toolchain, plugins for popular IDEs like Android Studio, IntelliJ, and VS Code, external monitoring providers such as Sentry, and DevTools - a collection of debugging and performance features. These tools offer various functionalities, including logging options, debug flags, UI inspection, network profiling, source-level debugging, memory analysis, and error tracking in production environments. By leveraging these tools effectively, developers can ensure the smooth functioning and optimal performance of their Flutter applications.

There is only so much control you can exert over your app's performance, but controlling the code is arguably most important. Sentry Performance provides code-level insights to help resolve performance bottlenecks. Query Insights allows users to identify slow database queries and their impact on API endpoints, enabling them to fix issues quickly. By analyzing query metrics, impacted API endpoints, and transaction details, developers can pinpoint performance regressions and make targeted updates to improve overall application performance.

Implementing lazy loading for images can improve performance, user experience, and Core Web Vitals such as Cumulative Layout Shift and Largest Contentful Paint. Lazy loading instructs the browser to load resources only when they are needed, preventing unnecessary data downloads and speeding up page load times. However, it's essential to carefully consider which images to lazy load, as loading images above the fold can negatively affect LCP scores. Adding width and height attributes to image elements can also prevent Cumulative Layout Shift for lazy loaded images. To monitor performance impacts of lazy loading, Sentry's JavaScript Browser SDK can automatically collect Core Web Vitals and send custom metrics. Configuring alerts in Sentry allows teams to quickly identify poor performance issues without sacrificing development time or CI/CD resources.

Sentry has rolled out new features to help developers quickly identify and prioritize high-priority issues, making it easier for them to pinpoint problems that require immediate attention. The company has introduced two new issue states: Regressed and Escalating, which provide more context into an issue's life stages and help users discover when fixed issues break or when ongoing or archived issues degrade. Additionally, Sentry has improved its Priority sort feature to display issues in order based on age, overall event volume, and recent event volume, making it easier for developers to find issues that need their attention fast.

The article explores the challenges and solutions related to optimizing memory usage and query performance in ClickHouse for Sentry's Session Replay product. Initially, the product faced significant memory constraints, making searches slow or impossible for large datasets. By restructuring query patterns and leveraging ClickHouse features such as preflight and data queries, materialized columns, and strategic indexing, Sentry was able to drastically reduce memory usage and improve query speeds. Additional methods included encoding columns to reduce disk size and implementing configuration settings to cap memory usage based on the number of unique aggregation keys. Performance benchmarking demonstrated a significant improvement in query throughput and latency, highlighting the effectiveness of these optimizations and making the system more scalable and efficient for handling large customer datasets.

A stack trace without source maps can be frustratingly difficult to decipher, but Sentry's ability to display original source code in stack traces can greatly improve debugging efficiency. To troubleshoot source map upload errors, it is essential to upgrade to the latest JavaScript SDK version and ensure that Debug IDs are being injected into the event payload. Additionally, verifying the source maps wizard configuration and checking for Debug ID injection tool capabilities can help resolve issues. If using release + abs_path, ensuring that the release and dist values match what was uploaded and verifying the file paths in the stack trace against the uploaded source map paths can also fix errors. Furthermore, Sentry recommends using Debug IDs for an easier setup process, as they uniquely identify and bind pairs of minified sources and source maps with a unique ID, allowing for more accurate mapping and debugging.

With Sentry, developers can quickly pinpoint and fix issues that impact their end users or business, and now Fly.io is integrating Sentry's application monitoring tools into its platform for a seamless experience. Fly.io provides the benefit of auto-instrumenting apps to start sending events to Sentry from the onset, making it easier for developers to get started with error monitoring. The integration creates a new Sentry project when an app is deployed through Fly.io, granting immediate access to Sentry's application monitoring tools and allowing developers to log in directly via the command line or their Fly.io credentials. This partnership aims to provide every developer with high-quality, affordable application performance monitoring, bringing them one step closer to fixing issues that impact end users or business.

Choosing the right metric for performance monitoring is crucial to ensure accurate insights and effective optimization. Averages provide a simple overview of overall performance but can be misleading when data is not uniform. Percentiles offer a more nuanced view by dividing data into segments based on their distribution, revealing how well the system is performing across different scenarios. The choice between averages and percentiles depends on the application's variability, user experience perspective, and monitoring objectives. By selecting the right metric and interpreting its results correctly, developers can gain a comprehensive understanding of their system's performance, leading to improved user satisfaction, faster issue resolution, and better scalability planning.

With the release of Sentry’s Session Replay tool, backend developers can now easily identify and prioritize issues that occur on the frontend, even if they don't directly experience them. Sentry's Session Replay records the user's browser session, but only shows issues that happen within that session. The tool is integrated with frontend errors and handles this case beautifully. By linking replays to backend endpoints via Backend Error linking, developers can get a complete end-to-end view of their APIs. To use Session Replay, developers need to set up the minimum required SDK version for their project and configure trace propagation. With Session Replay in action, developers can gain intimate insight into user experiences, identify priority issues, and accelerate their ability to fix problems.

Sentry values software freedom for users, but not necessarily in the same strident terms as the Free Software movement. They prioritize access to technology and knowledge for developers, as well as sustainability for maintainers. The company was founded by David Cramer in 2008 and has since become a successful business around the project, with hundreds of others contributing over the years. Sentry relicensed some core repos under the Business Source License (BUSL-1.1) to protect itself from exploitation, but now gives away significant amounts of money to non-commercial projects. The company is exploring ways to move forward in building a shared understanding of Open Source values and philosophy.

OpenTelemetry is an open-source observability tool that has become the second-largest project in the CNCF, behind Kubernetes. Its adoption can be driven by various reasons, some good and some bad. Poor reasons include following the crowd, building one's own observability product without considering the complexity involved, and vendor neutrality as a sole reason. Instead, good reasons include bridging gaps with manual instrumentation, sending data to multiple vendors, and gaining insight into third-party libraries and services. It is essential to approach technology choices pragmatically and evaluate use cases carefully before adopting OpenTelemetry or any other tool. The project's open-source nature makes it appealing, and contributions are always appreciated.

Sentry has released several updates in August, including the addition of rage and dead clicks analysis in Session Replay, profiling capabilities for more SDKs, a new Sentry + Discord integration, a VS Code extension for Codecov, and a feature to recommend events in Issue Details. These features aim to help developers better identify user frustration, optimize performance, streamline their workflow, and improve their overall development experience with Sentry.