Company
Date Published
Author
Nicolas Ruflin
Word count
630
Language
English
Hacker News points
None

Summary

With the release of Elastic 7.9, a new data stream naming scheme for time series data was introduced, structured to enhance the organization and management of indices and data streams in Elasticsearch. The scheme divides datasets into distinct data streams using a naming convention composed of three parts: type, dataset, and namespace, which are combined using a hyphen. This approach results in smaller, denser data streams, leading to improved space efficiency, faster queries, and more granular control over data management aspects like rollover, retention, and security. The scheme's flexibility allows users to organize data as needed, and it reduces the permissions required for data ingestion. Supported from Elastic Stack version 7.9 onwards, the scheme is implemented in Elastic Agent data and can be adopted by other data shippers by following the naming convention and incorporating the necessary data stream fields. Future blog posts will explore the technical workings and applications of this naming scheme in more detail.