April 2023 Summaries

Zephyr Archaeotech Emporium, a fictional company, is exploring ways to accelerate its development process using Pulumi, specifically for locally testing application code of their online store. In a conversation between Alice, a developer, and Bob from the platform engineering team, they discuss the challenges of the current code testing approach, which involves deploying an entire environment on AWS, thereby consuming significant time. Alice suggests using Pulumi's Docker provider to automate local testing, allowing her to quickly deploy the online store application to a local Docker daemon using TypeScript. This method enhances the inner development loop by enabling rapid iteration on code changes. Although Docker Compose is a viable tool, Alice prefers Pulumi for its ability to incorporate logic into her program, such as specifying whether to build a container or use an existing version, thus allowing selective testing of specific services. This approach not only speeds up development but also maintains consistency across environments since Pulumi is already used for deploying to Kubernetes in other stages. Alice demonstrates the efficiency of this process by making code changes and observing results quickly, emphasizing that while it doesn't replace the need for per-developer stacks, it provides quick feedback on code modifications before deploying to more extensive test environments.

Pulumi AI is a purpose-built AI assistant designed to streamline the creation of cloud infrastructure using Pulumi by leveraging Large Language Models and GPT technology. It addresses the complexity of modern cloud platforms, which include millions of configurable properties across numerous resources, by allowing users to generate tailored Infrastructure as Code (IaC) solutions through natural language queries. This tool enhances productivity by providing a starting point for cloud infrastructure projects, enabling users to interactively refine their solutions and automate deployments via the pulumi-ai CLI. Pulumi AI aims to elevate the abstraction level of cloud infrastructure management, offering higher-level components when available, and reducing the need to manually curate examples for specific use cases. The tool is positioned as a significant productivity booster for cloud engineers, with a focus on maintaining simplicity and composability in solving complex problems. Pulumi AI is already valued by thousands of users, with ongoing plans to expand its capabilities and impact in cloud engineering.

The blog post guides readers through creating a hub-and-spoke network architecture in AWS, focusing on centralized egress and traffic inspection using Pulumi, an infrastructure as code tool. It describes the process of setting up spoke VPCs as Pulumi component resources, testing centralized egress, and implementing traffic inspection with AWS Network Firewall. The post emphasizes the benefits of using Pulumi to manage complex infrastructure through real programming languages, facilitating the logical grouping of resources and efficient code organization. It also provides a practical example of testing centralized egress by deploying a sample workload and verifying that traffic routes correctly through the inspection VPC's NAT gateway. The post concludes by highlighting the cost-saving and policy enforcement advantages of a centralized hub-and-spoke architecture, inviting readers to learn more about Pulumi through workshops and community engagement.

Pulumi Insights is a significant enhancement to Pulumi's infrastructure as code platform, offering new capabilities in intelligence, search, and analytics for cloud infrastructure management. By leveraging advancements in generative AI and Large Language Models (LLMs), Pulumi Insights enables users to discover, learn, and interact with cloud resources across various providers like AWS, Azure, and Google Cloud, using familiar programming languages. The platform introduces Pulumi AI, an assistant that helps create cloud infrastructure through natural language input, and Pulumi Resource Search, which facilitates multi-cloud searches and provides detailed analytics regarding cloud resource usage. These features, available to all Pulumi Cloud users, allow for integration with business intelligence tools to generate insights on cloud infrastructure, enhancing productivity and enabling more efficient management of complex cloud environments.

AWS Lambda has evolved significantly since its introduction in 2014, expanding from basic cloud functions to a robust serverless platform. Despite its growth, Lambda's utility for data-intensive or latency-sensitive applications was hindered by a 6 MiB payload limit and the requirement for fully buffered responses before delivery. To address these limitations, AWS introduced Lambda Response Streaming, which allows partial responses to be sent to clients as they are ready, thereby reducing time to first byte (TTFB) and supporting larger payloads up to a soft limit of 20 MiB with a throughput of 2 MiB/s. This feature necessitates a new Lambda function handler signature that facilitates immediate data streaming to the client, supports NodeJS v14.x, v16.x, and v18.x, and includes an updated billing model based on bytes processed and streamed. While streamed responses are not compatible with API Gateway or Application Load Balancers, they can be accessed through Lambda Function URLs or the InvokeWithResponseStream API. AWS provides tools like Pulumi for packaging and deploying Lambda functions with streaming capabilities, thereby enhancing Lambda's functionality for modern microservices architectures.

The blog series introduces building a hub-and-spoke network architecture on AWS with centralized egress and traffic inspection, using Pulumi and Python. This architecture includes an inspection VPC, AWS Transit Gateway, and spoke VPCs, focusing on cost savings, stable public IP addresses, and centralized traffic inspection. The inspection VPC, the only VPC with internet access, routes all traffic through its NAT gateways, while the Transit Gateway facilitates inter-VPC connectivity and central routing. The series emphasizes security and isolation, noting that inter-spoke VPC communication is generally not enabled unless beneficial for shared production environments. Initial steps in creating this architecture involve setting up Transit Gateway resources and inspection VPCs, with the promise of further development in subsequent installments, including the creation of spoke VPCs and enhanced traffic inspection capabilities.

Pulumi has introduced project-scoped stacks to its DIY backends, aligning them with the Pulumi Service backend's method of managing stack names within projects. This change allows users to employ common stack names like "test" and "prod" across different projects without conflicts, eliminating the need for previous workarounds where stack names were globally unique across all projects. With the new release, users can upgrade existing DIY backends to the project-scoped format using the `pulumi state upgrade` command, but older versions of the Pulumi CLI will not recognize the new format. This update also enables streamlined referencing of stacks within the Pulumi CLI. While the Pulumi Service backend offers enhanced usability and security, the DIY backend provides users with more control over where state information is stored, supporting local filesystems and various cloud storage options. Pulumi's ongoing commitment to improving the Infrastructure as Code experience includes both the open-source SDK and CLI, along with Pulumi Cloud, which offers options ranging from free individual plans to tools for enterprise-level management.