Accelerating Apache Spark with Gluten + Velox: Vectorized Execution for Big Data at ScaleRemoved

Acceldata's ODP Spark with Gluten and Velox introduces a significant enhancement to Apache Spark's performance by shifting from a traditional row-based execution model to a vectorized execution model, addressing performance bottlenecks in large-scale data processing. This advancement is achieved through the integration of the Gluten framework and Velox execution engine, which process data in columnar batches, leading to faster query execution, reduced CPU cycles, and lower memory allocation pressure. The solution maintains full compatibility with existing Spark applications, requiring no changes, and demonstrates notable performance gains on TPC-DS benchmarks, offering enterprises substantial infrastructure savings and improved efficiency in CPU, memory, and I/O operations. With features like SIMD acceleration, improved CPU cache utilization, and NUMA-aware memory management, the system is poised as a next-generation execution engine for Spark analytics, with ongoing developments such as broader SQL function coverage, streaming support, and GPU acceleration integration.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.