June 2025 Summaries

SYNTHETIC-2 is a next-generation, open-source reasoning dataset and distributed inference platform built on a globally-distributed stack, powered by the DeepSeek-R1-0528 model, which enables contributions from a wide range of computing resources, from consumer GPUs to large-scale NVIDIA and AMD clusters. The platform supports pipeline-parallel distributed inference, allowing large models to be run on consumer devices through sequential stage processing. It employs the TOPLOC v2 verifiable inference system to ensure the integrity of results using locality sensitive hashing and reproducible Gumbel noises for sampling, making it robust across diverse configurations. The SYNTHETIC-2 dataset includes a comprehensive set of verified reasoning tasks that serve as high-quality training data for reasoning models and difficulty-annotated reinforcement learning data to improve model performance. This initiative aims to generalize reasoning skills beyond traditional tasks by aggregating data from various sources and designing diverse reasoning tasks. Following the launch, the next step is to use SYNTHETIC-2 as a foundation for the INTELLECT-2 distributed reinforcement learning run, which seeks to enhance model performance through globally distributed RL and introduce complex environments for coding and autonomous research tasks.