1BRC merykitty’s Magic SWAR: 8 Lines of Code Explained in 3,000 Words
Blog post from QuestDB
QuestDB, an open-source time-series database, excels in handling demanding workloads by offering ultra-low latency and high ingestion throughput, along with native support for Parquet and SQL. During the One Billion Row Challenge (1BRC), a remarkable optimization technique emerged, significantly enhancing Java code performance by parsing one billion temperature readings in less than a second. Quân Anh Mai's innovative solution, which utilized a technique known as "SIMD Within A Register" (SWAR), replaced the conventional approach of using if statements and loops with a sequence of 18 Arithmetic Logic Unit (ALU) operations, including bitwise shifts and arithmetic operations. This method efficiently parsed CSV input by aligning digits, transforming ASCII characters into digit values, and applying mathematical operations to convert these into accurate temperature readings, all without explicit conditionals, thus becoming a pivotal element in the top solutions of the contest.