January 2012 Summaries

In addressing the challenge of maintaining high availability in systems or services, the concept of "pressure release valves" is introduced as a pragmatic approach to managing potential catastrophic failures without over-engineering solutions. These valves act as adjustable mechanisms that can be activated during emergencies to mitigate the severity of problems while they are being resolved, such as configuration changes that disable critical functions or adjust operational parameters. The strategy involves brainstorming with team members to identify possible failure scenarios and devising temporary solutions to maintain partial functionality, thus reducing pressure on on-call staff during crises. While acknowledging that it's impractical to prevent every possible failure mode, the emphasis is on preparing for them through thoughtful discussion and planning, ensuring systems continue operating at a basic level even under duress.

PagerDuty, originally inspired by the founders' experiences with pager duty, has historically included pagers in its service ecosystem. The company has offered a feature that sends alerts as short emails, primarily for customers who use pagers, but this option was previously limited to select accounts. Due to increased customer demand, PagerDuty will now enable this feature for all accounts, allowing users to receive the same alert text via email as they would through SMS. However, the company cautions that while it does not restrict sending emails to an SMS gateway, it cannot ensure the timely or successful delivery of messages sent in this manner.

The author shares their positive experience at the Southern California Linux Expo in Los Angeles, emphasizing the benefits of attending more conferences outside San Francisco. They highlight engaging discussions on monitoring solutions, the presence of numerous open-source projects, and successful interactions with potential job applicants. The event, organized by Ilan and a volunteer team, was well-received, with an impressive turnout of over 2000 attendees. Despite missing some talks, the author notes the event's success in fostering customer contact and expects to return next year. The team also ran a "Be a PagerDuty Hero" contest, which included giving away an iPad, and distributed a significant amount of promotional items, underestimating the demand.

PagerDuty has introduced a new on-call scheduling tool that provides users with extensive flexibility to create various on-call rotations, such as recurring schedules and "Follow the Sun" shifts, by utilizing a layered approach similar to digital image editing. This tool allows for diverse scheduling configurations, from daily developer rotations to unique weekly shifts, and has been positively received by users. The tool incorporates features like smooth responsive JavaScript, non-blocking page loads, and dynamic calendar previewing, contributing to an enhanced user experience. As part of a broader effort to modernize their application stack and interface, PagerDuty is leveraging Twitter’s Bootstrap Framework to facilitate intuitive navigation and productivity. iCal and Webcal integration have also been added, with more updates anticipated in the future.