Introducing the enrich processor for Elasticsearch ingest nodes

Elasticsearch 7.5.0 introduced a new enrich processor for ingest nodes, allowing users to enrich documents with additional data from reference datasets during the ingestion process. Ingest nodes pre-process documents before indexing by using pipelines composed of various processors, each performing specific operations like splitting or removing fields. The enrich processor uses data from another index to enhance documents, such as adding a user's full name based on an email address match. For example, flight data can be enriched with airport data by creating an enrich policy that manages reference data efficiently, like indexing US airport codes to enrich flight information with airport-specific details. This policy creates a system index for quick document enrichment, and once executed, a pipeline can be defined to enrich and clean up documents by removing redundant fields and renaming others. The enriched data is then incorporated into the documents, replacing fields with enriched JSON objects containing additional information. This feature allows users to enhance their data workflows and is accessible by upgrading to Elasticsearch 7.5 or using Elasticsearch Service.