July 2018 Summaries

CircleCI's blog post highlights their internal use of workflows, specifically through their circleci-images repository, which is openly available to demonstrate the effectiveness of their workflows toolset. This repository is crucial for building and deploying Docker convenience images used by numerous customers daily, leveraging an efficient and reliable pipeline that incorporates features such as fan-in/fan-out, multiple execution environments, cron scheduling, and branch-based filters. The post outlines the process for creating these images, starting from a commit to deploying on Docker Hub, using a dual workflow approach to accommodate nightly builds and immediate feedback for code changes. The nightly workflow is scheduled to run on the master branch, whereas the regular commit workflow handles changes as they happen, with both workflows sharing similar job configurations to ensure simplicity and reusability. By utilizing YAML anchors and branch-based filtering, CircleCI optimizes its workflow for both staging and production environments, demonstrating the power of multi-platform workflows and the flexibility to choose appropriate execution environments for different jobs.

The text discusses advanced techniques for controlling workflows in job orchestration, specifically through the use of branch and tag filtering and approval options. It explains that filtering helps deploy or run tests selectively, based on defined criteria, to avoid unnecessary builds—particularly useful for managing large repositories or deploying from specific branches. The article provides examples from companies like DataDog, Google Cloud, Aeternity, and Azure, illustrating various implementations of these techniques. DataDog uses tags to manage release versions, Google Cloud employs regular expressions for tag matching, Aeternity schedules jobs at specific times to optimize resources, and Azure uses manual approval for jobs, especially in resource-intensive environments. These strategies help optimize workflow efficiency and resource allocation while maintaining precise control over job execution.

Multi-executor workflows provide the flexibility to run jobs across different platforms, such as Linux and macOS, and accommodate various programming languages within the same workflow, enhancing modularity and efficiency. These workflows allow teams to choose from multiple resource classes, enabling the configuration of CPU and RAM to match job requirements, which helps avoid memory constraints and optimize performance. Examples include Facebook React, which tests across Android, iOS, and tvOS platforms using different machine sizes to meet specific memory needs, and Envoy Proxy, which leverages multi-platform workflows to test across macOS and Linux while using varied resource classes to optimize resource usage. These approaches enable teams to perform simultaneous testing and build processes across multiple platforms, providing valuable feedback and ensuring that applications function correctly in diverse environments.

The text highlights the versatility of workflows in software development, focusing on how they provide teams with the flexibility to run builds and tests in various configurations, such as sequentially or in parallel. By examining real-world examples from open-source projects like Artsy and Mapbox, the text illustrates different strategies in job orchestration, including sequential workflows for incremental feedback and parallel workflows for faster testing feedback. It also discusses how to handle test failures, offering options to rerun entire workflows or just the failed jobs to maintain efficiency and continuity in the development process. The article is part of a series that aims to guide users through the implementation and benefits of using CircleCI workflows, offering insights into job orchestration, multi-executor control, and other advanced configurations.

GitHub Releases is a feature that enhances git tags by providing a detailed description, marking capabilities for releases, and a platform to upload and host release artifacts, such as binaries, on GitHub. The article explores the process of automating these releases using CircleCI, focusing on the use of the ghr command to upload artifacts to GitHub. It explains how to set up a CircleCI job, "publish-github-release," which can be triggered by tagged commits and utilizes a GitHub token for authentication. The post highlights the benefits of using GitHub Releases for distributing software conveniently and provides a technical walkthrough of integrating with CircleCI to streamline the release process. Additionally, it touches on the flexibility of choosing when to publish a release, using a SemVer-like regex to filter tags, while also providing a complete regex for SemVer for reference.