Company
Date Published
Author
Denis Rechkunov
Word count
1854
Language
-
Hacker News points
None

Summary

In Filebeat versions 8.10.0 and 7.17.12, a new fingerprint mode has been introduced for Filestream inputs, allowing users to identify files using a SHA256 hash of their content instead of relying on file system metadata. This change addresses issues with unstable file identifiers that can lead to data loss or duplication, particularly in environments where file systems cache and reuse inodes. The fingerprint mode offers a unique file identifier by hashing a configurable byte range of the file, providing a more stable solution compared to using device_id and inode values. While the new mode incurs a performance hit, optimizations made during its implementation have resulted in an overall performance improvement in the new Filebeat release, making Filestream faster even with the fingerprint mode enabled. This advancement highlights the ongoing focus on enhancing Filebeat's performance and reliability in handling file system events and data ingestion.