Company
Date Published
Author
Jorge Quilcate, Victoria Xia, Wade Waldron
Word count
1648
Language
English
Hacker News points
None

Summary

Apache Kafka-based applications use event logs to decouple producers and consumers, enabling choreographed service collaborations without direct communication between them. However, making sense of these communication patterns can be challenging due to implicit dependencies and the use of different programming languages, platforms, and teams. Distributed tracing is a method for recording, collecting, and recreating execution traces from distributed components, which can help analyze latency issues and debug applications at runtime. Tracing libraries offer instrumentation to collect interactions between components, making it easier to enable tracing. Zipkin is a popular distributed tracing project that provides a Java tracer library with built-in instrumentation for Kafka clients, streams, and connect pipelines. By adding interceptors to Kafka Connect plugins, tracing capabilities can be implemented, allowing for the collection of traces from every message produced and consumed by any application using the producer and consumer APIs. Tracing introduces performance impact but can be mitigated with sampling strategies. The use of distributed tracing has been instrumental in helping a Norwegian consulting firm, SYSCO AS, analyze latency issues and debug applications at runtime.