Scaling redirects to infinity on Vercel

Handling redirects at scale poses challenges related to latency and system costs, especially when managing millions of redirects. Vercel previously managed redirects via routing rules and middleware, which became inefficient as the number of redirects increased. To address this, a new approach was developed using Bloom filters and JSONL file format to manage redirect data. Bloom filters, which can quickly determine if a redirect might exist, reduce unnecessary processing by skipping the JSON parsing for most requests. To further optimize, the redirect data is sharded, allowing for smaller data slices to be processed only when necessary. This approach significantly lowers the memory usage and latency for redirect lookups, particularly by employing binary search over sorted keys instead of parsing entire JSON shards. This design reduces per-request CPU demands and eliminates latency spikes, making redirect handling faster and more efficient. Bulk redirects are now available for Pro and Enterprise customers, allowing for large-scale migrations and management of broken links with a capacity of up to 1 million redirects per project.