Zero to a Million: Instrumenting LLMs with OTEL
Blog post from Arize
Arize's OpenInference instrumentation has reached one million monthly downloads, marking a significant milestone in observability for AI using OpenTelemetry (OTEL). The journey has been challenging but rewarding as the team, along with other key players in the industry, paves the way for OTEL LLM instrumentation. They faced several challenges such as dealing with latent data, navigating lists in OTEL, avoiding attribute loss, handling futures and promises, managing streaming responses, and defining the scope of instrumentation. Despite these hurdles, Arize remains committed to shaping the future of standardization in LLM observability.
| Trend | Post Mentions | Total Month Mentions | Posts | Companies | MoM |
|---|---|---|---|---|---|
| OpenTelemetry | 14 | 365 | 40 | 19 | -1% |
| LLM | 12 | 3,598 | 465 | 143 | -7% |
| Real-time | 4 | 4,144 | 915 | 211 | +5% |
| Observability | 2 | 1,843 | 317 | 87 | +17% |
| Vector Search | 2 | 4,605 | 291 | 90 | +25% |
Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.