GPT-4 is Getting Faster 🐇

Recent observations indicate that GPT-4 is rapidly closing the latency gap with GPT-3.5, showing a significant decrease in processing delays over the past few months. Although specific prompt characteristics can affect response times, the overall trend demonstrates improving speed for GPT-4, with median request latencies for both models remaining under 1 ms per token. Notably, in the 99th percentile, GPT-4's latency has more than halved in just three months. While GPT-4 is still more expensive, it no longer lags behind in speed for most requests. Additionally, there's an ongoing investigation into whether latency increases as users approach their rate limits, suggesting a potential deliberate slowdown by OpenAI.