We test our system's behavior before attempting a refactor, using combinatorial and differential testing techniques to ensure bug-for-bug compatibility between versions. This approach allows us to capture any discrepancies, even as the refactor runs in production. We also use shadow testing, which involves passing customer requests to both the existing version and the refactored version, to catch regressions before exposing them to customers. To control rollout, we use feature flags, which enable us to selectively ship changes to customer accounts or specific customers, allowing for faster deployment and rollbacks. Observability is also crucial, as it enables us to publish information about customer requests, such as latencies and error rates, to detect any issues in production. By combining testing, rollout control, observability, and feature flags, we can de-risk large-scale, potentially high-impact refactors and protect the customer experience.