December 2019 Summaries

Pulumi experienced significant growth and development over the past year, marked by a 15x increase in its customer base and the release of Pulumi 1.0, which introduced major improvements in quality and compatibility without breaking changes. The company expanded its global community through numerous events and workshops, while also launching a roadmap for Pulumi 2.0, focusing on productivity and security for enterprises. Notable advancements included the introduction of Pulumi Crosswalk for simplifying cloud infrastructure patterns, the preview of CrossGuard for policy as code, expanded language support with .NET, and enhanced identity and secrets management options. Pulumi also offered more state storage solutions, additional CI/CD integrations, and tools to ease adoption for teams transitioning from existing infrastructure, resulting in a robust framework supported by an extensive range of providers. The year 2019 was transformative for Pulumi, setting the stage for further growth and innovation in 2020.

Pulumi CrossGuard is a policy-as-code solution designed to validate infrastructure policies at deployment time, ensuring compliance with security and reliability best practices. It allows organizations to express policies as code using programming languages, which can prevent the creation of out-of-compliance resources. CrossGuard provides two main types of policies: ResourceValidationPolicy and StackValidationPolicy. ResourceValidationPolicy focuses on individual resources, preventing their creation or modification if they violate set policies, while StackValidationPolicy assesses the entire stack of resources after they have been created or modified, allowing for checks that require existing resource states. These policies are integrated into Pulumi's workflow, running during the pulumi preview and pulumi update commands to evaluate cloud resource definitions. The flexibility in writing these policies enables users to enforce rules either on a per-resource basis or across an entire stack, depending on their specific needs. Pulumi CrossGuard aims to enhance infrastructure management by catching potential issues early in the deployment process.

AWS Lambda cold starts, which occur when a worker must be assigned to a request, present a significant challenge for latency-critical serverless applications. The introduction of Provisioned Concurrency at AWS re:Invent 2019 provides a solution by allowing users to maintain a specified number of always-warm workers dedicated to particular Lambda functions, thus mitigating cold start delays. Despite the benefits, Provisioned Concurrency introduces additional costs, as AWS charges per hour for provisioned capacity regardless of usage, and requires management to maintain optimal concurrency levels. Dynamic provisioning options, such as autoscaling based on scheduled profiles or utilization, offer flexibility but may face initial performance issues. While these methods help manage cold starts, they involve careful balancing between minimizing latency and controlling costs. The article emphasizes that high-load functions benefit from a stable level of provisioned concurrency, and latency-sensitive APIs, particularly with Java and .NET runtimes, require a strategic approach to concurrency management to balance performance and expenses effectively.

Pulumi has expanded its .NET support to include Kubernetes infrastructure management, allowing developers to use C#, F#, and VB.NET alongside existing support for TypeScript and Python. This integration provides advantages such as strong typing, rich IDE support, and the ability to use familiar .NET languages and APIs. Developers can abstract common Kubernetes functionalities into reusable components, enhancing code reusability and infrastructure design. The platform also allows the deployment of custom Docker images within Kubernetes clusters, supporting seamless application and infrastructure deployment and versioning. Furthermore, Pulumi's compatibility with major cloud providers like AWS, Azure, and GCP enables the creation and management of both managed Kubernetes clusters and associated applications using a unified programming model. The release signifies a significant enhancement in Pulumi's .NET offerings, with ongoing improvements anticipated.

AWS Elastic Kubernetes Service (EKS) offers multiple options for managing and scaling Kubernetes clusters, including Fargate, Managed Node Groups, and manually managed EC2 instances, each providing varying levels of control and complexity. Fargate simplifies the process by automatically handling the provisioning, scaling, and scheduling of worker nodes, allowing users to focus on Kubernetes pods without managing the underlying infrastructure. Managed Node Groups offer a middle ground, providing automated scaling with some configuration control, while manually managing EC2 instances allows for the highest level of customization at the cost of increased complexity. The integration of EKS with AWS services like IAM, EBS, and CloudWatch enhances cluster management by aligning with existing AWS security and monitoring practices. The Pulumi platform further simplifies EKS deployment by offering infrastructure as code capabilities, enabling users to easily provision and manage EKS clusters with minimal code.

AWS recently introduced the IAM Access Analyzer, a tool that uses automated reasoning to identify insecure access to AWS resources such as S3 Buckets and Lambdas. Simultaneously, Pulumi launched CrossGuard, a policy-as-code solution that validates policies during deployment. The integration of these two services enhances security by combining IAM Access Analyzer's ability to detect access issues with CrossGuard's capability to flag and prevent insecure deployments in real time. By implementing CrossGuard policy packs alongside IAM Access Analyzer, users can enforce best practices for AWS resources, preventing common security pitfalls like public-read access to S3 Buckets. The collaboration between these services is designed to quickly identify and rectify security vulnerabilities at deployment time, thus reducing the risk of costly mistakes. This early-stage integration demonstrates how combining innovative tools can advance infrastructure security, encouraging users to explore policy as code with open-source resources.

Pulumi's 2.0 roadmap introduces enhancements aimed at improving productivity and catering to the needs of modern cloud architectures, with an emphasis on maintaining platform stability and compatibility. The update focuses on two main themes: enhancing productivity through support for more languages, tools, and testing capabilities, and providing robust features for teams and enterprises, including advanced security, compliance, and flexible hosting options. Key features include the introduction of "CrossGuard" for policy as code, improved support for Go and .NET Core languages, and the "watch mode" for continuous deployment from the IDE. Pulumi 2.0 also offers preview features that users can try by activating the experimental mode, and it encourages community feedback to refine the platform further. The roadmap reflects Pulumi's commitment to supporting modern cloud applications and infrastructure in a cohesive manner, with ongoing improvements driven by user input.

Pulumi's vision is to bridge the gap between application developers and infrastructure teams in the cloud by applying successful software engineering practices to cloud infrastructure management, thus promoting creativity and rapid iteration. With the introduction of the pulumi watch command in version 1.5, developers can experience real-time updates to their cloud infrastructure by automatically deploying changes upon saving code, allowing for quick experimentation and learning. This feature is particularly beneficial for serverless and Kubernetes environments, where rapid iteration is crucial. Pulumi watch complements existing tools by facilitating faster development cycles without replacing the need for careful production deployments. Additionally, the integration of infrastructure and application code in development helps dissolve traditional silos, enhancing efficiency and agility in cloud application deployment. Pulumi watch is currently available in preview, aiming to further improve iteration speed, logging capabilities, and integration with other modern infrastructure scenarios.

Pulumi CrossGuard is a Policy as Code solution that enables the enforcement of custom business and security rules across organizations, allowing administrators to apply these policies universally or to specific stacks. With its open-source nature, CrossGuard supports all Pulumi users, including those with the Individual Edition, while offering advanced policy management to Enterprise customers. It allows for the creation and enforcement of policies written in TypeScript or JavaScript, but applicable to stacks in any language, thus ensuring compliance before resource creation. By empowering developers and operators to self-provision infrastructure, CrossGuard facilitates the codification of best practices, helping organizations maintain security, compliance, and cost-efficiency. The platform's flexibility is illustrated through the use of Policy Packs, which can prevent the creation of non-compliant resources, such as public-read AWS S3 buckets, thereby enhancing security and operational standards. CrossGuard's preview version is accessible to all Pulumi users, with plans to expand its policy SDKs to other languages, and feedback from users is encouraged for continued improvement.