How Grafana Labs Is Running Jaeger at Scale with Prometheus and Envoy

Grafana Labs has been optimizing query performance on its metrics platform by integrating Jaeger for distributed tracing, significantly enhancing query speeds by up to 10 times. Initially challenged by poor query performance at scale, they utilized Prometheus to pinpoint slow services and environments, allowing for targeted investigation and improvements through Jaeger's UI. They faced challenges with load balancing in Jaeger, which were resolved by implementing gRPC changes and using Envoy for better traffic distribution. Additionally, Grafana Labs developed a Jaeger mixin to enhance monitoring, which they have contributed to the community. They are working on future improvements, including implementing exemplars in Prometheus to facilitate direct navigation from dashboards to specific traces and exploring scalable tail-based sampling with OpenTelemetry Service to optimize data storage and tracing efficiency.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.