How to Use Roboflow Batch Processing on Images Stored in AWS S3
Blog post from Roboflow
Roboflow Batch Processing offers a cost-effective solution for processing large volumes of images stored on AWS S3, providing a 25-times cheaper alternative to the Hosted API while maintaining control over automation and storage. The process involves generating signed URLs using AWS CLI to secure data access, creating JSONL reference files, and triggering Roboflow workflows directly from the AWS environment to run inference on large image datasets. Users need to have their images stored on AWS S3 and a Roboflow Workflow prepared for the data they wish to process. The procedure includes generating signed URLs for images, creating a batch reference file, executing a processing job with a Roboflow Workflow ID, and exporting results in JSONL or CSV format after completion. This setup allows for the automation of image processing directly from S3, managing over 100,000 images in a single batch job, and receiving webhook notifications for full pipeline automation. The guide provides detailed steps and suggests consulting Roboflow Batch Processing documentation for further learning.