May 2021 Summaries

Amazon's Elastic Container Service (ECS) has evolved significantly since its inception in 2014, offering various deployment options for containerized applications. Initially, ECS required managing EC2 instances, but with the introduction of Fargate in 2018, it shifted towards a serverless model, allowing containers to run without infrastructure management. The latest advancement, ECS Anywhere, enables users to deploy containers on nearly any server, whether it's an EC2 instance, a virtual machine from another cloud provider, or a bare metal server in a private data center, while still utilizing ECS's control plane for deployment management. This flexibility addresses various needs, including latency, regulatory, and operational requirements. The setup involves creating AWS infrastructure, deploying nodes with a custom script, and using ECS to manage container deployment with a new launch type, EXTERNAL. Pulumi, an infrastructure as code platform, streamlines this process, supporting multi-cloud deployments and allowing cloud engineers to manage infrastructure with familiar programming languages. This integration facilitates seamless management of container images, ECR registries, and cloud resources, enhancing the ability to deploy and update applications efficiently.

The text discusses the benefits and process of using AWS App Runner, a service that simplifies deploying containerized applications without the need for extensive configuration of virtual networking components such as load balancers. It highlights the use of Pulumi, an infrastructure as code platform, to set up the necessary infrastructure for using App Runner, specifically focusing on building a container-based application with Amazon's Elastic Container Registry (ECR). The text provides a detailed walkthrough of creating an ECR repository, setting up roles and permissions for image access, and deploying the application using Pulumi's automation capabilities. It emphasizes the ease and efficiency of using AWS App Runner and Pulumi for deploying applications, allowing developers to focus on application development rather than complex infrastructure setup. The process is illustrated through a demo video, showcasing the deployment and running of a simple application, underscoring the streamlined approach to application deployment facilitated by App Runner and Pulumi.

Pulumi offers a framework for testable Infrastructure as Code, enabling fast local unit testing without cloud dependency, although it initially lacked comprehensive IAM Policy support. The author addresses this by developing a library to manage IAM Policy documents using JSON objects, noting the absence of built-in validation which prolongs the feedback loop when errors occur. Emphasizing the importance of compliance in the financial industry, the author explores various methods for testing and validating IAM Policies, including leveraging Pulumi's aws.iam.getPolicyDocument API and the AWS CDK, despite limitations such as the need for mock functions and the complexity of integrating the entire CDK package. The @thinkinglabs/aws-iam-policy library is introduced as a solution for writing and validating policies as code, significantly reducing syntax errors and improving the feedback cycle. The text concludes by discussing future improvements such as enhanced condition validation and support for additional policy elements, illustrating the benefits of extending Pulumi programs to preemptively address errors and enhance developer guardrails in policy deployment.

Skai, a global marketing platform, transitioned its core services to AWS to accommodate business growth and overcome the limitations of its private cloud infrastructure. To streamline this migration and manage the complexity of its cloud infrastructure, Skai adopted Pulumi as its Infrastructure as Code (IaC) tool, allowing for a programmatic and modular approach to deployment. The migration aimed to harness AWS's managed services, such as RDS, Lambda, and EKS, to enhance scalability and efficiency. Skai's DevOps team played a crucial role in the migration, leveraging Pulumi's support for cloud-native architectures and Python programming to ensure a smooth transition. The organization structured its IaC projects to facilitate multi-team collaboration, allowing different teams to manage specific infrastructure components while maintaining alignment with overarching goals. Additionally, Skai implemented dynamic providers within Pulumi to integrate internal metadata systems, ensuring consistent registration states for new application instances. The transition to AWS, supported by Pulumi, enabled Skai to better support its innovation and growth demands while maintaining a loosely-coupled yet tightly-aligned approach to infrastructure management.