Launch Week II - Day 1: Announcing the Batch Scrape Endpoint
Blog post from Firecrawl
Firecrawl's second Launch Week introduces the new Batch Scrape Endpoint, designed to streamline web scraping tasks by allowing users to scrape multiple URLs simultaneously, enhancing efficiency and time management. This endpoint functions like the existing /crawl endpoint but supports processing multiple URLs in one job, offering both synchronous and asynchronous methods to suit different workflow needs. Users can initiate the Batch Scrape with a simple cURL command, specifying URLs and desired output formats like Markdown or HTML, and receive results immediately or retrieve them later using a job ID. The Batch Scrape Endpoint improves efficiency by reducing network overhead, provides flexibility in processing methods, and allows customization of output formats to fit diverse data workflows. As part of Launch Week II, this feature is the first of several new capabilities aimed at enhancing web scraping effectiveness, with Firecrawl seeking user feedback to refine and tailor their services further.