Integrating Apache Kafka Clients with CNCF Jaeger at Funding Circle Using OpenTelemetry

Funding Circle employs Kafka as a central element in their event-driven microservices architecture, which offers benefits like decentralization but also presents challenges in system understanding and observability. To address these challenges, they explore distributed tracing, which profiles and monitors applications to identify failures and performance issues and provide insight into system flows. The blog post discusses the state of tracing in the Kafka ecosystem and introduces OpenTelemetry tools for enhancing telemetry collection without modifying source code. Funding Circle demonstrates a scenario utilizing OpenTelemetry for distributed tracing across Kafka-enabled services, highlighting the benefits of improved observability in debugging, incident response, and documentation. Despite challenges in stateful stream processing that hinder full end-to-end observability, the company remains optimistic about future improvements in the community's tracing capabilities. The post concludes by recommending starting with non-intrusive telemetry using OpenTelemetry to enhance productivity and gradually exploring more customizable options as familiarity with the tools grows.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.