Content Deep Dive
Disk Spilling in a Vectorized Execution Engine
Blog post from Cockroach Labs
Post Details
Company
Date Published
Author
Alfonso Subiotto Marques
Word Count
1,547
Language
English
Hacker News Points
48
Summary
CockroachDB has added disk spilling to its vectorized execution engine, enabling it to execute memory-intensive queries when there is not enough memory available. The new feature allows the engine to fall back to disk for certain types of queries, improving analytical capabilities in CockroachDB. Disk spilling is implemented using a single building block that supports various on-disk algorithms, including sorts, hash joins, and merge joins. With this addition, the experimental_on vectorize mode has been renamed to on, signifying that the vectorized execution engine is now ready for production use.