Home / Companies / OpenObserve / Blog / Post Details
Content Deep Dive

How to Send uberAgent Data to OpenObserve with Nginx and njs

Blog post from OpenObserve

Post Details
Company
Date Published
Author
Chaitanya Sistla
Word Count
3,201
Company Posts That Month
5
Language
English
Hacker News Points
-
Post removed?
No
Summary

uberAgent’s Elasticsearch data-stream bulk format can appear to ingest successfully into OpenObserve while storing no records because it places the target index in the URL rather than in each bulk action line, whereas OpenObserve requires an `_index` field on that line. The proposed solution uses an Nginx proxy with the njs JavaScript module to read incoming bulk request bodies, extract the index from the URL, inject it into `create` or `index` action lines, and forward the modified data to OpenObserve. The setup requires correctly loading njs, ordering Nginx locations so bulk requests reach the shim, preserving the index segment in uberAgent’s receiver URL, and setting `client_body_buffer_size` at least as high as `client_max_body_size` so request bodies are not spooled and lost to the script. Validation should focus on populated bulk response items, destination record counts, and the appearance of the named stream rather than HTTP 200 status codes alone. After ingestion works, users should map uberAgent’s actual event-time field to OpenObserve’s required `_timestamp` field to ensure accurate timelines, and should adjust the platform’s accepted historical-ingestion window when backfilling older data.

Trends Found in this Post
Trend Post Mentions Total Month Mentions Posts Companies MoM
Observability 9 472 102 54 -85%
OpenTelemetry 7 125 18 15 -83%
Use This Data

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.