Speeding up Pulumi Operations by up to 20x
Blog post from Pulumi
Pulumi has introduced a journaling feature aimed at significantly enhancing the speed of cloud infrastructure operations by up to 20 times while maintaining data integrity. Traditionally, Pulumi operations involved creating and storing snapshots of the cloud infrastructure state, a process that was both resource-intensive and slow due to serialized uploads, especially for large stacks. The new journaling method circumvents this by sending incremental changes, or journal entries, to the backend, allowing for faster and parallel processing without compromising data integrity. Benchmarks demonstrated substantial time reductions in setting up complex infrastructure, exemplified by AWS and Pulumi app setups. This performance boost is presently accessible behind a feature flag, with Pulumi users invited to test by enabling the PULUMI_ENABLE_JOURNALING environment variable. The feature is expected to roll out broadly by January, marking a step towards optimizing Pulumi's default operational efficiency.