Value Separation in Pebble: Storage Engine Optimization

CockroachDB's version 25.4 introduces value separation within its key-value storage engine, Pebble, enhancing compaction efficiency and potentially increasing throughput by up to 50% for certain workloads. This improvement is achieved by separating values from keys during storage, reducing redundant I/O operations and thus lowering costs. The concept, inspired by prior research like the WiscKey paper, helps optimize the balance between read performance, space amplification, and write amplification. Pebble's implementation involves storing separated values in blob files, which allows for more efficient compactions by primarily handling smaller value handles instead of large data chunks. However, this approach introduces tradeoffs such as potential read I/O overhead and space amplification due to fragmented references, which are managed through blob file rewrite compactions. These compactions reclaim space by moving only still-referenced values, maintaining performance while controlling storage utilization. Although value separation offers clear benefits for workloads with large values, CockroachDB also applies this technique to smaller values in latency-tolerant regions to minimize write amplification without compromising read performance. Future developments may focus on balancing the competing needs of supporting efficient historical data scans and reducing unnecessary I/O, potentially leveraging blob file design to optimize data placement across different storage tiers.