July 2022 Summaries

Anthony Accomazzo discusses the challenges of efficiently detecting changes in APIs, particularly focusing on the limitations of webhooks and the necessity for a robust method of consuming changes, which is a common usage pattern. He critiques webhooks for their delivery issues and lack of support for comprehensive updates, proposing the use of an /events or /updated endpoint as a more reliable solution. The /updated endpoint would allow consumers to paginate through updates using a combination of updatedAfter and afterId parameters, which help avoid record omission or duplication errors common in other pagination designs. Accomazzo suggests this approach would ensure completeness and reliability in tracking changes, enabling consumers to initialize their cursors and replay records easily. He also explores the benefits of using pagination tokens over manual cursor management, although he acknowledges their opacity. Additionally, he proposes a unified /updated endpoint for listing changes across all tables, which could be filtered by record type to suit different consumer needs, and highlights the potential need for a /deleted endpoint to track deletions.

Sequin offers a solution for developers looking to integrate HubSpot data with their Postgres databases, which can be challenging due to HubSpot's Webhooks API limitations. Webhooks are useful for subscribing to events without exceeding API rate limits but have drawbacks, such as limited retries and insufficient event coverage for certain CRM objects. HubSpot's Webhooks API only supports Contacts, Companies, and Deals, leaving out other CRM types and custom objects, and each subscription tracks only one property change, necessitating additional polling for comprehensive updates. Workflows, while offering broader object support, are not a full replacement for Webhooks due to their limitations and enrollment restrictions. An alternative approach is optimized polling using HubSpot's CRM search endpoints to manage data changes without exceeding rate limits, though this can miss intermediate updates. Sequin simplifies these processes by handling the complexities of syncing HubSpot data to Postgres, including managing cursors, rate limits, and schema conversion, allowing developers to maintain up-to-date data without the need for extensive manual configurations.