Shipping Logs

Fly.io is a platform that runs apps globally with just a few commands, generating a lot of log output. Centralizing these logs is important for various use cases such as debugging, tracking, collating, correlating, coalescing, and condensing the happenings of your code into human-parsable information. Fly.io's logging system works by grabbing stdout from processes run in apps and sending it to a centralized location via NATS (a "fancy, clustered pubsub service"). Flaps, a proxy that sits in front of NATS, ensures users only see their own logs. To ship logs, an instance of the Fly Log Shipper can be run, which configures a Vector sink and runs Vector to send logs to various destinations like Loki, Datadog, or Cloudwatch. The process is similar regardless of the log aggregator used.