Heap utilizes Apache Spark to enhance its Heap Connect product, which facilitates rapid data transfer to customer warehouses, but encountered performance issues due to a slowness in Spark's ORC file reader. The problem arose from an exponential algorithmic issue in processing binary trees, leading to significant delays in query planning within Spark's Driver component, causing Executor idling. The team addressed this by restructuring their tree representation to avoid deep, skewed trees, resulting in substantial performance improvements. They circumvented the need for immediate changes to Spark's internals, thereby accelerating their fix deployment, and ultimately contributed a solution to Spark, eliminating the exponential complexity in ORC predicate conversion, which was incorporated into the Spark 3.0 release.