Remove the Noise to Cure Alerting Fatigue

At the Nagios World Conference North America, Arup Chakrabarti from PagerDuty discussed effective alert management in production systems, emphasizing the importance of filtering out non-actionable alerts to focus on those impacting customer experience. The proliferation of alerting due to cheaper computing and automation can lead to alert fatigue, making it crucial to discern which alerts truly matter, particularly those related to customer-facing issues like website availability for e-commerce. Chakrabarti suggests analyzing alert history to evaluate incident severity and implementing a tiered alert system, such as tagging alerts with severity levels, to ensure only critical issues disturb engineers during off-hours. By managing alerts effectively, organizations can maintain a sense of urgency for significant problems and reduce the mean time to resolution, allowing engineers to rest assured that non-critical alerts will be addressed during regular hours, thus combating alert fatigue.