Benchmarking fluent-bit with Clickhouse

Hermes, currently in its prototype phase, is an audit logs service utilizing a Go REST API server to ingest logs and send them to Loki, with the goal of finding scalable solutions for high traffic without data loss. Various combinations of ingesters like Vector and Fluentd, and storage tools such as MongoDB and ClickHouse were benchmarked to determine their suitability for Hermes. Testing was conducted on a MacBook Pro with dockerized tools, revealing that Fluent Bit, despite its fast log ingestion capabilities, does not natively support ClickHouse, necessitating custom plugins. ClickHouse demonstrated exceptional querying performance, efficiently managing both short-term high loads and long-term light loads. The benchmarks indicated Fluent Bit's capability to handle up to 2000 requests per second, although performance declines with larger batches. Future blogs will provide further insights as Hermes and other development tools evolve.