Taming A Game-Changer: Honeycomb and GraphQL at VendHQ

Evan Shaw, Lead Engineer at vendhq.com, describes the transition to using GraphQL for Vend's APIs and the subsequent challenges and solutions in monitoring server performance. GraphQL was praised by frontend engineers for its flexibility and structured schema compared to RESTful APIs, but it introduced complexities on the backend, particularly in aggregating meaningful metrics due to its single HTTP endpoint that could handle numerous query types. Initial attempts to monitor performance with Datadog and AWS X-Ray proved inadequate; however, the implementation of Honeycomb tracing, using the Go library gqlgen, provided better insights, though it initially resulted in traces with excessive segments. By refining the tracing approach to only include complex fields, Shaw identified and addressed inefficiencies such as unnecessary queries and ineffective dataloader configurations, significantly improving internal request handling. Despite ongoing challenges with trace focus and retention, the transition has enhanced Vend's ability to understand and resolve server-side issues, encouraging further exploration and optimization of production environments.