On-Demand Dagger Engines with Argo CD, EKS, and Karpenter

In this blog post, the authors describe how to leverage Argo CD, AWS EKS, and Karpenter to deploy self-hosted GitHub Actions runners and Dagger Engines in a way that streamlines Dagger upgrades in production. The approach is specifically built for horizontal scaling, which means Dagger Engines are ephemeral processes that show up and go away based on queued pipelines. This allows for reduced costs as usage decreases. The authors also discuss the trade-offs of horizontal vs vertical scaling, where vertical scaling involves "monolithic" long-lived Engines that run most of the pipelines. The setup described in the post is currently being used by the Dagger team for all their pipelines, including those from contributions to Dagger. It achieves significant cost savings and improves efficiency in CI job scheduling and Dagger Engine stopping. The authors encourage others to try out this new approach and provide feedback on any issues encountered. They are also working on the third version of Dagger in production and experimenting with running GitHub Actions Runner as a service within Dagger.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.