Scaling Autonomous Site Reliability Engineering: Architecture, Orchestration, and Validation for a 90,000+ Server Fleet

Cloudways, a leading managed PHP hosting service, faced significant challenges in managing over 90,000 servers and handling a growing volume of support requests, which led them to implement AI-based Site Reliability Engineering (SRE) agents to enhance their operations. The AI-powered Cloudways Copilot significantly reduces the burden on support teams by providing faster and more consistent insights for web application troubleshooting compared to human agents. The system employs a monitoring layer to detect anomalies, an orchestration layer to execute commands securely across servers, and a control plane that routes alerts for analysis. The integration of the DigitalOcean Gradient AI Platform has been pivotal, offering a reliable, flexible infrastructure that supports both open-source and proprietary models, streamlining the deployment and scaling processes. Cloudways also implemented a dual validation approach involving manual reviews and a secondary AI agent to ensure output quality, mitigating risks associated with diverse application environments. The system focuses on tasks that benefit most from AI, such as identifying server resource issues and tracing excessive requests, while balancing AI's strengths and limitations to maximize operational efficiency.