How Perk cut recovery time to 31 minutes with centralized rollbacks in CircleCI

Perk, previously known as TravelPerk, significantly improved its incident recovery process by implementing CircleCI’s rollback pipeline feature alongside the Platform Team Toolkit, enabling faster and more reliable rollbacks across dozens of services. By transitioning from outdated and fragmented CLI tools to a centralized rollback control plane within CircleCI, Perk reduced mean time to resolve (MTTR) during incidents, achieving rollbacks in under four minutes for core applications and reducing MTTR to 31 minutes for high-priority incidents. The new system replaced bespoke CLI commands with just four CircleCI config files, facilitating organic adoption across over 80 projects without requiring extensive training. This transformation not only improved rollback speed and confidence but also reduced maintenance burdens on the DevOps team by substituting custom script maintenance with centralized configuration management. Looking ahead, Perk is exploring similar strategies for frontend projects and deploying to lower environments to further enhance its CI/CD processes.