Great user experiences start with reliable applications, and catching failures before they reach users is key. LangSmith has launched Alerts to help monitor LLM apps and agents in real-time, supporting alerts based on metrics like error rate, run latency, and feedback scores. Proactive monitoring is crucial for LLM-powered applications due to their reliance on external services and the quality of their output, which can be impacted by unpredictable changes in prompts or models. LangSmith Alerts provides an early warning system for these issues, allowing developers to focus on specific subsets of runs with filters and aggregation windows. The feature supports alerts via PagerDuty or custom webhooks, enabling seamless integration into existing workflows. Future updates will add more types of alerts, relative value thresholds, and custom time windows, making LangSmith Alerts an increasingly powerful tool for observability.