Span or Attribute in OpenTelemetry Custom Instrumentation

OpenTelemetry tracing offers a powerful way to enhance visibility and correlation in distributed systems by efficiently adding attributes to spans, which are units of work, to encapsulate critical event data. Attributes, such as request parameters and feature flag values, provide a cost-effective method for enriching trace information, particularly when using providers like Honeycomb, which charge per event. While spans are preferred for marking significant operations, especially across network or asynchronous boundaries, logs can still be useful for capturing fixed-time events or exceptions without overwriting existing attributes. The strategy for structuring traces, whether by adding attributes or creating new spans, should align with the desired level of detail and correlation needed for observability, allowing developers to effectively tell the story of their application's performance and behavior.