May 2024 Summaries

In this blog post, the authors describe how to leverage Argo CD, AWS EKS, and Karpenter to deploy self-hosted GitHub Actions runners and Dagger Engines in a way that streamlines Dagger upgrades in production. The approach is specifically built for horizontal scaling, which means Dagger Engines are ephemeral processes that show up and go away based on queued pipelines. This allows for reduced costs as usage decreases. The authors also discuss the trade-offs of horizontal vs vertical scaling, where vertical scaling involves "monolithic" long-lived Engines that run most of the pipelines. The setup described in the post is currently being used by the Dagger team for all their pipelines, including those from contributions to Dagger. It achieves significant cost savings and improves efficiency in CI job scheduling and Dagger Engine stopping. The authors encourage others to try out this new approach and provide feedback on any issues encountered. They are also working on the third version of Dagger in production and experimenting with running GitHub Actions Runner as a service within Dagger.

Dagger has successfully completed a SOC 2 audit, achieving certification in five key areas: Security, Availability, Processing Integrity, Confidentiality, and Privacy. This rigorous certification demonstrates Dagger's commitment to privacy, security, and operational excellence. The company partnered with Vanta to automate its internal compliance program and implemented a self-service trust center for customers to access its security controls and policies. With this milestone, Dagger now has a CISO function through a partnership with Barr Advisory, solidifying its dedication to providing secure and reliable software to its customers.

The author of this text is a developer who has been working on improving notifications in their pipeline using the Dagger notify module, which allows sending notifications to Slack and Discord from a Dagger pipeline. This was popular among community members, with one implementation even being adopted by a company evaluating Dagger for production use. The author then learned about Novu, an open-source notifications infrastructure platform that makes it easy to programmatically manage transactional notifications to multiple destinations with a simple cross-language API. Inspired by the success of the Dagger notify module, the author created a Novu module using the Novu Go library, which allows sending notifications to Slack, Discord, email, and SMS with a single call to the Dagger CLI. The module is still in development, but it has the potential to greatly simplify notification management for developers.

Emmanuel Sibanda, a software engineer, was plagued by the "works on my machine" problem for his application's unit tests. He found that using Dagger Functions in his production CI pipeline resolved this issue and provided quicker feedback, consistent CI environments, and improved pipeline maintainability. By leveraging Dagger Functions, Emmanuel can now run parts of his test pipeline locally with consistency, allowing for faster feedback and reducing development friction from inconsistent and slow CI environments.