Custom Elasticsearch Aggregations for Machine Learning Jobs
Blog post from Elastic
Rich Collier's article explores the process of using custom Elasticsearch aggregations to manage derivative calculations in machine learning jobs, particularly focusing on detecting sharp rates of change in data, such as a "brown-out" recovery. The article emphasizes setting up a job configuration using the ML API in Kibana and creating a datafeed that defines the data source, which includes aggregating data through a date histogram and performing a sum aggregation followed by a derivative calculation. This process results in the creation of a new field called "orders_deriv" that is used by the machine learning job to detect anomalies, such as a significant recovery in order volume. Although the results can be viewed in Elasticsearch's UI, the article notes a limitation in visualizing derivatives over time due to constraints in the current Single Metric Viewer, which cannot dynamically reverse-engineer complex queries for raw data plotting. The author suggests that future updates may address this limitation while also recommending existing resources for machine learning users to enhance their workflows.
No tracked trend matches for this post yet.
Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.