Automatically scale self-hosted runners in AWS to meet demand

Self-hosted runners in CircleCI enable teams to customize and control their CI/CD infrastructure by hosting their own scalable execution environments in a private cloud or on-premises. These runners offer flexibility for teams with unique security or compute requirements and can be set up quickly, providing access to popular CircleCI Cloud features such as parallelism, test splitting, and SSH debugging. To manage fluctuations in resource demands and avoid unnecessary costs from idle resources, teams can implement an auto-scaling solution using AWS Auto Scaling groups, which adjusts the number of self-hosted runners based on job queue depth. This tutorial demonstrates setting up an auto-scaling solution for CircleCI's self-hosted runners with AWS, involving the creation of a launch template, Auto Scaling group, IAM policy, and role, along with the use of AWS Lambda functions to monitor job queue depth and adjust runners accordingly. By leveraging these tools, teams can maintain efficient and cost-effective CI/CD pipelines, adapting the infrastructure to meet dynamic workload requirements while maintaining full control over execution environments.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.