Why rate limits can't stop distillation attacks
Blog post from WorkOS
Distillation attacks can use large numbers of low-volume API accounts to collect model outputs and hidden reasoning traces for training competing systems, making per-key rate limits ineffective when account creation is cheap. OpenAI reported a July 2026 campaign involving 16,000 extraction-pattern requests across more than 4,000 users and linked a core cluster to individuals associated with Moonshot AI, while Anthropic described wider alleged campaigns involving millions of Claude exchanges and varying patterns of quiet, distributed use and high-volume activity. The attacks reportedly exploited replayable encrypted reasoning blocks that could be transferred between sessions and reconstructed, though providers said the activity did not involve breaking encryption or directly accessing stored conversations. OpenAI and Anthropic responded by closing replay vulnerabilities, limiting reasoning output, detecting extraction patterns, restricting fraudulent accounts, and strengthening signup, identity-verification, and network-monitoring measures. The central argument is that effective defense requires resolving related accounts into shared actors through signals such as device profiles, payment methods, network behavior, and prompt patterns, while using verification challenges rather than automatic cluster-level bans to reduce harm to legitimate organizations with shared infrastructure.
No tracked trend matches for this post yet.
Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.