Four Considerations When Designing Systems For Graceful Degradation

There are limits to any system, and when these limits are exceeded, unexpected behaviors can occur, leading to services degradation and outages. To plan for macro-level issues like saturation, long latency, and excessive workloads, SRE and DevOps teams can design for graceful degradation of services, allowing for more limited functionality while avoiding catastrophic failure. This can be achieved through shedding workload, time shifting workloads, reducing quality of service, or adding more capacity to the system. By considering these strategies and designing for resiliency, teams can create more resilient services that can handle spikes in workloads without affecting uptime, ultimately preparing for the inevitable and unexpected patterns in their systems.