May 2024 Summaries

GitHub Actions is presented as an accessible CI platform for small companies, while Buildkite is argued to better meet the needs of organizations with roughly 75 or more engineers as test suites, CI workloads, costs, and security requirements grow. The comparison emphasizes Buildkite’s simpler AWS-based self-hosting model, which uses EC2 agents and avoids the Kubernetes, Docker-in-Docker, webhook, monitoring, and security-management complexity associated with GitHub Actions Runner Controller. Buildkite is also described as offering stronger reliability, audit and permission controls, faster monorepo cloning through local Git mirrors, and built-in test analytics that identify passing, failing, flaky, slow, and unreliable tests. Its first-class configurable automatic retry feature can reduce disruption from intermittent failures while providing visibility into retried steps, whereas GitHub Actions generally relies on third-party retry actions and external tools such as Datadog for comparable observability. Blacksmith concludes by positioning itself as seeking to close these enterprise CI capability gaps while allowing teams to remain within the GitHub ecosystem.

Blacksmith announced that it has achieved SOC 2 Type 1 compliance following an independent audit of its internal controls, reinforcing its stated commitment to security and data protection for its CI platform. The company also reports hiring an independent security consultant for penetration testing and implementing disaster-recovery measures for its databases and backend infrastructure. Blacksmith says it retains only job-execution metadata, its GitHub app cannot access customer secrets, and it uses single-use just-in-time GitHub tokens to limit credential exposure. GitHub Actions workloads run in isolated, ephemeral Firecracker virtual machines with hardware isolation, memory-safe components, and state destruction after each job. The company plans to continue using independent audits and consultants to assess security and offers its SOC 2 Type 1 report upon request.