About the Tailscale.com outage on March 7, 2024

On March 7, 2024, tailscale.com experienced a 90-minute outage due to an expired TLS certificate, largely affecting its marketing materials and documentation, but leaving most user operations unhindered. The incident was traced back to a major website refresh and migration to a new hosting provider in December 2023, which included a unique IPv6 proxy configuration that was deemed a misconfiguration by the provider, preventing automatic certificate renewal. Although probers were in place to check certificate expirations, they only checked over IPv6 and missed the impending expiry. To resolve the issue, Tailscale temporarily removed the "extra" AAAA records and manually renewed the certificates, restoring access swiftly. Moving forward, Tailscale plans to address the root issue with redundant calendar alerts for manual renewals, update their prober infrastructure to separately check IPv4 and IPv6 endpoints, and improve IPv6 support to prevent future disruptions. Despite the outage, Tailscale's design principle of enabling direct connectivity between machines ensured that the majority of users were unaffected.