Company
Date Published
Author
Eric Thomas
Word count
2346
Language
-
Hacker News points
None

Summary

Highlight has developed a logging-as-a-service product utilizing Clickhouse to enhance its error monitoring capabilities by seamlessly integrating logs with other debugging tools. Initially designed to offer detailed insights into user actions leading to errors, Highlight's logging service allows users to efficiently search logs with custom attributes and auto-injected debugging data. The company adopted OpenTelemetry's schema, leveraging Clickhouse for its efficient querying capabilities, and incorporated multi-tenancy by adding a ProjectId column. Challenges such as high memory usage due to timestamp precision were overcome by transitioning to second precision, significantly improving performance. The service employs cursor pagination using a combination of timestamps and UUIDs to ensure stable navigation through logs. Highlight's use of Go for backend development, along with structured query building, facilitates efficient data handling, while the supportive Clickhouse community and robust documentation have been instrumental in the product's development and future scalability plans.