Tracing Makes Concurrency Bugs Easy to Spot

A subtle concurrency bug was identified in a TypeScript function that handles requests to a /win endpoint, where the function saves a win record and returns the total winnings. The bug occurred because the save operation and the query to calculate total winnings were running concurrently, causing the returned total to exclude the most recent win. The fix involved adding an await statement to ensure the save operation completed before executing the query. This concurrency issue was initially elusive, passing tests intermittently and behaving differently in production compared to local environments. The use of tracing tools like Honeycomb helped visualize the concurrent operations, highlighting the bug and emphasizing the importance of such tools in improving expertise in concurrent programming. The trace was generated using auto-instrumentation for Express, TypeORM, and Postgres, with a recommendation to integrate tracing into the development process for better handling of asynchronous code.