February 2023 Summaries

Zephyr Archaeotech Emporium, a fictional online retailer specializing in rare arcane artifacts, is transitioning from a monolithic to a containerized architecture using Kubernetes to better manage their growing online presence. To handle this complexity, they are adopting Pulumi for managing both infrastructure and application deployments, which allows them to utilize a familiar programming language and achieve fast, repeatable deployments. The blog post explores key considerations for storing Pulumi code, such as whether to maintain it in the same repository as application code or in a separate one, which depends on organizational needs like access control and code management. Zephyr opts for a monorepo approach, with a single team managing both infrastructure and application code, and uses Pulumi stacks to handle multiple environments, initially creating a production stack and a development stack. This is the first installment in a series that will detail how Zephyr's Pulumi practices evolve with their growth, focusing on best practices for infrastructure as code, version control, and the use of Pulumi stacks to manage multiple instances of their application.

Event streaming is crucial for industries needing real-time data processing, and Apache Kafka stands out as the leading open-source platform for this purpose. Confluent Cloud offers a managed solution for Kafka, providing features like elasticity, security, and monitoring, and enables hybrid cloud scenarios by linking on-premises producers and consumers. By utilizing the Confluent Cloud Pulumi provider, users can efficiently create and manage Kafka clusters, topics, and service accounts through infrastructure as code, facilitating the quick deployment of event-driven applications. The blog post demonstrates setting up a Kafka cluster on Confluent Cloud with Pulumi, highlighting the ease of initializing projects, adding resources, and testing the infrastructure, thereby emphasizing the operational simplicity and enhanced functionality achievable with this approach. Through Pulumi's integration, organizations can manage Kafka resources using programming languages, allowing them to focus on delivering value while leveraging Kafka's capabilities.

FinOps, as defined by the FinOps Foundation, is a cloud financial management discipline that facilitates collaboration among engineering, finance, technology, and business teams to make data-driven spending decisions, thereby maximizing business value. This approach integrates financial best practices into the development process, encouraging developers to be mindful of resource usage at various stages, ultimately leading to significant operational expense savings and better growth forecasting. In smaller organizations, developers often manage cloud costs directly, while larger entities typically assign this responsibility to a central IT or Cloud Platform team. The practice of FinOps involves not only reactive optimization, where organizations review and adjust costs based on bills, but also a proactive approach that provides real-time feedback on how architectural changes impact costs. Pulumi, an infrastructure as code platform, facilitates both reactive and proactive cost management by enabling the design of guardrails for resource provisioning, continuous drift detection, and policy enforcement. Implementing a Tag Policy is essential for cost allocation, and automation tools such as Pulumi's CrossGuard and Automation API further enhance governance by enforcing organizational standards and enabling streamlined processes. The adoption of FinOps practices can lead to substantial financial optimization through strategic sourcing and discount negotiations with cloud vendors, ensuring that cloud operations align with business objectives and financial constraints.

An IT self-service "vending machine" in an enterprise setting allows employees to efficiently request and access pre-approved cloud resources, with Pulumi programs orchestrating these resources behind the scenes. The example discussed involves using Pulumi to create an AWS child account within an AWS Organization, which can be initiated via a Pull Request in a git repository. This setup is beneficial for both large enterprises and smaller teams, promoting the organization of AWS accounts using the least-privileged access principle. The organizational structure described includes an Organizational Unit (OU) with member accounts, where user accounts for developers are provisioned in the root account, allowing them to assume specific roles in target accounts. The process involves creating IAM roles and backup policies to ensure compliance and resource backup, with Pulumi providing a flexible programming model to manage these infrastructures. The approach encourages structuring AWS accounts to enforce least-privileged access, offering a self-service mechanism that is especially advantageous for organizations seeking SOC2 certification or those requiring strict resource isolation.