Data-driven CI pipeline monitoring with pytest

Tinybird utilizes pytest as a crucial component of its CI/CD pipeline, enhancing testing and performance monitoring to ensure fast and reliable code deployment. With over 3,000 tests and frequent code updates, Tinybird emphasizes the importance of CI/CD observability to maintain agility and confidence in shipping code rapidly. By implementing a data-driven approach and integrating additional observability tools, Tinybird reduced CI execution times by over 60%, helping to identify and address bottlenecks and flaky tests. The company developed the pytest-tinybird plugin to facilitate real-time analytics and visualization of CI metrics, utilizing Tinybird's platform for capturing, analyzing, and automating CI data. This approach allows for improved resource allocation and test execution order, contributing to a more efficient CI pipeline. The plugin, open-sourced for public use, is designed to send pytest data to Tinybird, enabling users to harness SQL for insights and publish metrics as APIs, which can be visualized in tools like Grafana and Datadog to monitor CI health and performance.