Improving data ingestion: better error feedback

Tinybird emphasizes the importance of providing a seamless data ingestion experience, particularly for users dealing with large datasets. The company has introduced improvements to its ingestion process, aiming to make it easier to handle billions of rows reliably and flexibly without sacrificing speed. Key challenges in data ingestion include dealing with incoherent data types, unexpected null values, and missing columns, which can be difficult to detect in large volumes of data. Tinybird offers multiple methods for data ingestion, such as importing via URL or uploading local files, each with distinct processes for job tracking and error feedback. Enhancements have been made to the API and UI to provide clearer insights into ingestion errors, such as parsing issues or rows sent to quarantine, thus facilitating easier troubleshooting. These initial improvements are part of a series of updates aimed at further simplifying the ingestion process, with future posts to explore additional advancements like fine-tuning guessing mechanisms and enhanced control over quarantine tables.