Bring the whole web to your apps and agents with Apify webhooks

Apify enables developers to turn websites without APIs into callable Actors, packaged scraping or automation tools that can run from its console, API, schedules, or agents, with results stored in datasets, key-value stores, and request queues. Because datasets are append-only and lack flexible querying, sorting, updates, or joins, applications that need to use fresh scraped data typically fetch it after a run and store it in their own databases. Run-completion webhooks provide this notification mechanism for scheduled or on-demand Actors, sending a compact payload containing the run object and dataset ID rather than the scraped data itself. The recommended implementation verifies a shared-secret header, quickly acknowledges the delivery, fetches and validates the run through Apify’s API, confirms account ownership and successful status, retrieves dataset items, and processes them idempotently in a worker to handle retries and duplicate deliveries. Apify does not sign webhooks, so additional safeguards include IP allow-listing, avoiding secrets in URLs, and validating each referenced run; Hookdeck Event Gateway can add edge authentication, HMAC signatures, queueing, rate limits, and best-effort deduplication. For long-running individual requests, ad-hoc webhooks can be attached to a specific Actor run so apps or agent services can resume work when it succeeds, fails, times out, or is aborted, avoiding prolonged open connections or repeated polling.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.