Using Conductor Forks to run tasks in parallel - creating multiple images

The blog post discusses the implementation of a Netflix Conductor workflow designed to handle image processing by using a FORK operation to generate multiple image versions in parallel. The workflow takes an image as input and creates two different formats, jpg and webp, using separate parallel processes that run asynchronously to ensure efficiency. This demonstration illustrates the use of the FORK task to split workflows into multiple streams, allowing simultaneous execution of tasks before synchronizing the results with a JOIN task. The necessary tasks such as image conversion, resizing, and uploading to AWS S3 are defined and managed within the Conductor environment. The workflow and tasks, along with sample code, are available in the orkesworkers GitHub repository, enabling users to replicate and test the process locally. The post highlights the flexibility and speed of parallel processing in workflows and concludes with a teaser for an upcoming post that will explore using subworkflows instead of parallel tasks.