How to maximize span ingestion while limiting writes per second to a Scylla backend with Jaeger tracing

At Grafana Labs, the integration of Scylla, an open-source Cassandra-compatible backend, with Jaeger tracing has demonstrated impressive performance capabilities, handling approximately 175,000 writes per second despite being deployed with default settings on Google's Kubernetes Engine. The disparity between the high number of writes and the lower span ingestion rate, around 8,000 spans per second, is attributed to the indexing process in Jaeger, which writes multiple rows per span to support detailed search capabilities. To manage backend load while maximizing span ingestion, Grafana Labs recommends adjusting sampling rates using Jaeger's remote sampler and utilizing configuration options to control indexing, thereby reducing unnecessary writes. Implementing a tag blacklist, disabling certain indexes, and using "firehose" mode for spans with less indexing needs have been effective strategies, resulting in a significant reduction in backend writes. Overall, leveraging these configurations has allowed Grafana Labs to maintain high performance and stability in their distributed tracing infrastructure.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.