Content Deep Dive

How to build a machine data lake

Blog post from Axiom

Post Details
Company
Date Published
Author
-
Word Count
2,284
Company Posts That Month
4
Language
English
Hacker News Points
-
Post removed?
No
Summary

Axiom presents a machine data lake as a way to retain and query logs, traces, metrics, and security events without splitting them across aging systems or making older data difficult to access. Its design separates event datasets from purpose-built metrics datasets, recommends distinct OpenTelemetry datasets for logs, traces, and metrics, and uses correlation groups to connect related data; security events are stored in an OCSF dataset with documented attributes typed as columns and unfamiliar attributes preserved in an unmapped field. Schemas are derived from incoming data, allowing structured OpenTelemetry records to become queryable at ingestion while query-time parsing supports older unstructured data. Axiom argues that columnar compression makes retaining all events affordable, citing reductions of about 25 times—and up to 50 times for some structured data—rather than relying on sampling. Data remains accessible through one query path as it moves from a write-ahead log into compressed object-storage blocks and automatically shifts among access tiers, while serverless query compute is kept separate from storage. The approach can also coexist with Splunk workflows, enabling teams to add high-volume or previously uncovered security sources before deciding whether to expand.

Trends Found in this Post
Trend Post Mentions Total Month Mentions Posts Companies MoM
OpenTelemetry 12 822 142 56 -4%
Serverless 2 833 236 92 -40%
Observability 1 2,390 523 156 -23%
Platform Engineering 1 1,295 290 84 -36%
Use This Data

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.