Faster Query Processing: CPU Time

Starburst has implemented significant performance enhancements to improve query processing by focusing on increasing CPU utilization and reducing query wall time. A major change involves updating Trino’s query execution policy to default to a phased approach, which schedules only the stages of a query that can make progress, thereby reducing latency. Additional improvements include adaptively setting task concurrency to match the number of physical cores on a node and increasing the hive.split-loader-concurrency to enhance the processing of partitions and small files. Internal benchmarking showed an average 13% reduction in wall time, with improvements as high as 50% for specific queries. These enhancements are available in the upcoming LTS release and require minimal user intervention, as they leverage existing configurations to deliver faster query processing out of the box.