Being on-call can be challenging for engineers, often resulting in disrupted sleep and alert fatigue due to frequent, noisy alerts. To address this issue, an AI-powered On-call Assistant Bot was developed to handle alerts by performing an initial triage and responding only to significant signals. Integrated into Slack, the bot assesses alerts, providing context and recommended actions for genuine incidents, while muting unnecessary noise to prevent future disturbances. The workflow involves several components, including AI agents and automation tools like n8n, to seamlessly connect and manage alerts. The project showcases the potential of combining AI with observability workflows to reduce engineer burnout and improve efficiency. Through real-world examples, the bot demonstrates its ability to distinguish between signal and noise, gaining trust by explaining its decisions. The initiative also highlights a growing trend among organizations to develop AI solutions that alleviate operational burdens, with future plans to enhance the bot's capabilities further.