Company
Date Published
Author
Chris Wanstrath
Word count
729
Language
English
Hacker News points
None

Summary

After experimenting with various queue solutions, including Ara Howard's Bj, the team transitioned to Shopify's delayed_job (Dj) to address performance challenges associated with Bj's design decisions, such as spawning only one worker per machine and reloading the Rails environment for every job. These limitations led to inefficiencies, particularly with CPU usage, as the Rails startup process consumed significant resources. Dj, in contrast, allows multiple workers per machine and operates efficiently without the burden of repeated Rails startups, making it a suitable choice for handling up to 20 workers on a dedicated machine. This transition was facilitated by the use of a simple interface, RockQueue, which abstracted vendor-specific APIs and minimized changes to application code. The move to Dj was swift and effective, enhancing the speed and reliability of the queue system.