Darwin V6: Diagnostic-Guided Evolutionary Model Merging
Blog post from HuggingFace
Darwin V6 introduces a novel approach to AI model merging by using diagnostic-guided evolutionary algorithms to optimize the combination of two parent models at the tensor level. Unlike conventional tools that apply a uniform ratio across all tensors, Darwin V6 performs a Model Diagnostic Scan (MDS) that includes static tensor analysis and functional probing to determine each layer's importance, leading to per-tensor optimal merge ratios. The evolutionary algorithm optimizes the merging process by determining the best transplant intensity, resulting in improved performance metrics in benchmarks such as GPQA Diamond and ARC-Challenge. This innovative system ensures that superior tensors from one model are directly transplanted without interpolation, preserving the strengths of each parent model while enhancing the merged model's capabilities. The Darwin V6 engine is accessible for users to perform their own diagnostic-guided model merging, and several models under the Darwin family are available for public use, showcasing improvements over their parent models in various benchmarks.