Futureproofing Tines: Fair share orchestration

Tines, a platform handling over a billion automated actions weekly, has restructured its resource orchestration approach to address the challenges of fair-share resource distribution in a multi-tenant environment, ensuring equitable access and preventing system overload. The company transitioned from a per-story token bucket approach to a more robust system focusing on deterministic concurrency gates and continuously accruing per-story budgets. This new method includes a concurrency gate that caps the fraction of total workers consumed by any action starts, and a budget system that limits the number of actions a story can initiate. These changes aim to maintain predictable performance during high-demand periods and prevent any single story from monopolizing resources. The platform uses Sidekiq, PostgreSQL, and Redis to manage workflows, and has implemented measures such as temporary dampening of parallelism to prevent synchronized spikes. Despite the improvements, Tines is exploring further enhancements, such as batch database writes and dedicated queues for I/O-heavy actions, to address database write amplification issues and continue scaling efficiently.