My Journey Optimizing a CUDA Kernel with Polar Signals

During a Polar Signals hackathon, the author tackled creating a CUDA kernel to optimize string decompression, a task that served as an introduction to GPU programming and performance optimization. The project focused on implementing a kernel for decompressing FSST-encoded strings, a necessity for Vortex, Polar Signals' new file format, which aims for high-throughput decompression and query execution on GPUs. The initial kernel performance lagged behind CPU-based decompression, prompting the use of GPU profiling to identify and address memory-related bottlenecks. Key optimizations included memory load/store improvements, such as aligning loads and utilizing shared memory, though some attempts like reducing bank conflicts proved less effective. Eventually, optimizing memory stores yielded a significant performance boost, surpassing the CPU implementation. Further refinements included the split kernel optimization from the GSST paper, which enhanced execution efficiency by balancing workload across threads decompressing variable-length strings. Despite challenges, the project resulted in substantial improvements and highlighted the importance of understanding GPU architecture for achieving high performance. The final FSST CUDA kernel implementation achieved noteworthy throughput, and the experience underscored the potential of GPU profiling for future optimizations.