Increasing Memory Efficiency with an Improved Elastic APM Internal Intake Protocol
Blog post from Elastic
An improved version of the Elastic APM intake protocol has been developed to enhance memory efficiency and predictability for APM agents and servers. The original protocol relied on a simple, HTTP- and JSON-based system, where agents buffered transactions in memory before sending them to the APM server, potentially leading to memory inefficiencies. The new protocol draws inspiration from the Elasticsearch Bulk API, using newline-delimited JSON (NDJSON) to allow agents to serialize and stream events directly to the server without extensive buffering, thus simplifying memory management. This change enables agents to handle events with predictable memory usage, as each transaction and span is treated independently, reducing the need for large, memory-intensive HTTP request bodies. Initial tests suggest that this method not only optimizes memory usage on both the agent and server sides but also improves throughput, although certain issues remain to be addressed before full implementation in Elastic APM.
No tracked trend matches for this post yet.
Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.