September 2021 Summaries

The AWS Native provider, now known as the AWS Cloud Control Provider, has been released in preview for Pulumi users, offering robust support for AWS services with same-day feature updates through the AWS Cloud Control API. This provider is designed to provide a comprehensive and consistent interface for cloud infrastructure as code, offering faster updates and extensive coverage compared to other platforms. It allows Pulumi users to leverage the full suite of AWS resources and third-party resources in the CloudFormation Registry, ensuring a reliable provisioning lifecycle across all Pulumi-supported languages such as TypeScript, Python, Go, and C#. Additionally, the release includes the cf2pulumi tool for converting existing CloudFormation templates to Pulumi programs, while maintaining interoperability without relying on deploying CloudFormation templates. The AWS Native provider is automatically updated nightly with new AWS features, ensuring users have immediate access to new capabilities, and it can be used alongside the classic Pulumi AWS provider to enhance existing and new use cases. The provider aims to expand resource coverage and integrations with AWS services, with general availability planned for the following year.

Pulumi has relaunched its public roadmap, now utilizing GitHub Issues to enhance transparency and collaboration for its open-source tools, including the Pulumi CLI, SDK, and Service. This roadmap aims to provide users and contributors with insights into upcoming developments and potential areas for contribution, emphasizing the importance of community feedback in shaping future features. The roadmap details planned work for the next quarter and includes a backlog of issues, with most items sourced from GitHub issues in Pulumi's open-source repositories. Users can explore the roadmap by theme to understand specific areas of development, such as advancements in Native Providers or core infrastructure-as-code capabilities. Pulumi encourages users to engage with the roadmap by voting, asking questions, or expressing interest in contributing to the features listed.

In late 2018, AWS introduced its first EC2 instances utilizing ARM-based AWS Graviton Processors, which were designed for enhanced performance and cost-efficiency. Following the initial launch, AWS continued to develop its Graviton technology, culminating in the launch of the Graviton Challenge in June 2021 to encourage users to migrate their applications to the more advanced AWS Graviton2 processors. These Graviton2 instances offer up to 20% lower costs and up to 40% better price performance compared to x86-based instances. AWS has now expanded Graviton2 support to AWS Lambda Functions, allowing users to benefit from the improved price performance of the Graviton2 architecture when executing Lambda Functions. To deploy a Lambda Function on Graviton2 architecture using Pulumi, users must specify the architecture as "arm64" to ensure compatibility with ARM-based infrastructure. The text provides a detailed guide on creating the necessary IAM roles and deploying Lambda Functions and Layers on this architecture, highlighting the importance of cross-compiling application code for ARM64 in compiled languages like Go.

Pulumi has introduced a new REST API that enhances its already extensible cloud engineering platform, designed to interconnect and automate modern cloud systems. This API, which has been in use internally to power the Pulumi SDK, CLI, and Console, is now fully documented and available for public use, offering functionality to manage projects, stacks, cloud resources, policies, and more. The platform supports custom integrations and scenarios, enabling users to build unique solutions such as custom dashboards, metadata-driven automation projects, and enhanced security responses. Authentication and authorization are managed through Pulumi's identity and RBAC systems, ensuring secure access to manage teams, users, and audit logs. The REST API complements the Automation API and Webhooks, allowing users to efficiently query and manipulate cloud resources, making it a powerful tool for creating innovative cloud solutions.

The Cloud Engineering Summit 2021 features a diverse array of sessions focused on the Build track, curated by Kat Cosgrove, which emphasizes creating cloud applications and infrastructure using Modern Infrastructure as Code with general-purpose programming languages. The sessions include Kris Nóva discussing the drawbacks of Turing complete config management in "Infrastructure as Software," Sam Cogan advocating for reusable modules in "Reuse, Don’t Repeat," and Joep Piscaer highlighting the complexities of Kubernetes in "Kubernetes isn't the answer, but what was the question again?" Liz Fong-Jones will explore effective data insights in distributed systems through "What is OpenTelemetry?" while Steve Giguere offers practical guidance on securing deployments in "Hands-on: Taking a K8s Deployment from Default to Secure." Rahul Sharma delves into the development process and cultural shifts at AWS in "How AWS architects APIs for hyper scale," and Heidi Waterhouse concludes with a focus on inclusive web design in "The Future Should Be Uneven," emphasizing the universal benefits of accessibility. Attendees are encouraged to explore the full range of sessions and register for the summit.

Pulumi has introduced several updates and new features, enhancing its infrastructure-as-code capabilities and expanding its ecosystem with new cloud providers and resources. Key updates include a new Helm Release resource for Kubernetes that offers the same behavior as the helm CLI, as well as new providers for MinIO and Snowflake, allowing users to manage a broader range of resources. Additionally, Pulumi has released a series of updates to its Azure Native provider, adding 15 new resources, and has enhanced its CLI and core technologies with features like the pulumi about command for troubleshooting, hiding secret values in config settings, and enabling custom names for dynamic providers in Python. The Automation API in .NET now supports plugin installation options, and a schema checker for Pulumi Packages has been introduced. Pulumi Service improvements now ensure organizational invites direct collaborators to the correct identity provider, streamlining the sign-up process.

Cloud Engineering Summit 2021 is set to explore the core pillars of cloud engineering, focusing on the Build, Deploy, and Manage tracks, with a particular emphasis on automating and managing infrastructure in the Deploy track. This track aims to unify cloud-based systems through automated, auditable processes, thereby reducing human error and enhancing quality. Featured speakers include Adora Nwodo from Microsoft, who will discuss managing multiple environments using standard software engineering practices; Alyssa Miller from S&P Global Ratings, who will highlight security enhancements through threat modeling; Peter McKee from Docker, who will demonstrate sharing infrastructure changes via continuous previews; Andy Dang from WhyLabs, who will address data integrity and observability with whylogs; Angel Rivera from CircleCI, who will delve into ARM compatibility for cloud deployments; and Jenna Pederson from AWS, who will focus on scaling best practices for consistent and secure cloud resources. The event will also feature Evan Bradley and Tom Carrio from Dynatrace, sharing real-life team experiences in implementing these practices. The summit promises insightful discussions and practical solutions for cloud engineers.

Pulumi has introduced a new Snowflake Provider, allowing users to seamlessly integrate and manage their Snowflake resources alongside their existing infrastructure. This addition simplifies the process of setting up cloud storage and managing connections to Snowflake, catering to both current users and those considering adopting Snowflake. The provider enables various functionalities, such as configuring cloud storage as an external stage in Snowflake, integrating storage with Snowpipe, creating service users, and managing keys through preferred secrets managers. Comprehensive documentation is available for developers using Python, Go, TypeScript, and C#.

Pulumi has introduced a new Helm Release resource in its Kubernetes Provider and SDK, beginning with version 3.7.0, which improves upon the existing Helm Chart component by integrating the full feature set of Helm directly into Pulumi. This new resource manages orchestration and lifecycle hooks through Helm itself, providing advantages such as configurable await support and Helm state management, allowing users to interact with installed resources via the Helm CLI. While the Helm Release resource offers native configuration options for installing Helm charts, it has limitations in terms of inline transformation support and visibility within Pulumi's state. The existing Helm Chart component still offers a richer experience for those needing to manipulate charts or enforce resource policies through Policy-as-Code. Pulumi encourages users to explore this new resource during its public preview and provide feedback, while cautioning against its use in critical infrastructure due to its evolving nature.

The Cloud Engineering Summit 2021, focused on the themes of Build, Manage, and Deploy, offers a variety of sessions in the Manage track that emphasize managing cloud applications and infrastructure with Policy as Code, visibility, and access controls. Matt Stratton, the track chair, introduces sessions such as Joy Ebertz's exploration of authorization models, Quintessence Anx's insights into DevSecOps and secure incident response, and J. Paul Reed's discussion on Netflix's use of Risk Radar for forecasting incidents. Additionally, Fariba Khan and Stephen Van Gordon will delve into infrastructure as code with a focus on Pulumi, while Charles Xu will discuss multi-cloud Kubernetes operations. Jason Yee will present on navigating system failures, and Tim Banks will address the pressures of organizational speed in "Toxic Velocity: Speed Kills." These sessions aim to equip attendees with strategies for securely managing and developing cloud infrastructure and applications.

The blog post discusses the benefits and implementation of managed Kubernetes services, particularly focusing on Google's Kubernetes Engine (GKE). Managed Kubernetes services relieve users from the complexity of setting up and maintaining Kubernetes by handling tasks like load balancing, monitoring, and scaling, making it easier and more reliable than self-managed alternatives. The post provides a step-by-step guide to deploying a Kubernetes cluster on Google Cloud using Pulumi, Node.js, and npm, detailing the process of setting up a cluster, creating a namespace, deploying an NGINX application, and configuring a LoadBalancer service. It highlights the ease of scaling and maintaining applications on managed services and concludes with instructions on using kubectl to interact with the cluster. The post hints at future discussions on deploying Kubernetes on other cloud providers like Amazon EKS.