Company
Date Published
Author
James Baiera
Word count
879
Language
-
Hacker News points
None

Summary

Elasticsearch for Apache Hadoop (ES-Hadoop) 5.0.0 offers significant updates and enhancements that improve stability, introduce new features, and ensure compatibility with newer versions of integrations. Key updates include dropping support for older versions of Hive, Storm, and Spark to improve the codebase, while adding compatibility with Hive 1.0, Storm 1.x, and Spark 2.0. The HDFS Repository now integrates directly with Elasticsearch, eliminating the need for disabling the JVM SecurityManager. Additionally, the release introduces the Scroll Slicing functionality for increased parallel computing, native support for Spark Streaming to address connection resource limitations, and the ability to specify ingest pipelines and nodes to optimize data flow. The release also addresses various bugs, enhancing the overall performance and reliability of the software.