Airflow in Action: Boosting BI at Visa. 100% Trust in Data With 70% Lower Overhead

Chinni Krishna Abburi, a Business Intelligence Engineer at Visa, presented a guide at the Airflow Summit on how Visa transitioned from manual BI processes to an automated system using Apache Airflow, drastically improving efficiency and data trust. Previously, Visa's BI team struggled with a traditional setup that became inadequate as data volumes increased, leading to delayed dashboard refreshes and loss of stakeholder trust. By implementing Airflow, Visa established an end-to-end automated, reliable, and scalable data pipeline that integrates with modern tools like Databricks, AWS, Tableau, and Power BI. This transformation reduced manual intervention by 70%, decreased data refresh times from 24 hours to under 2 hours, and ensured zero missed SLAs over six months, achieving 100% data trust. The session highlighted lessons learned, such as starting small, prioritizing Dag readability, implementing early monitoring, establishing ownership, and treating Dags with software engineering best practices. Airflow has become the backbone of Visa’s BI operations, providing operational confidence and scalable infrastructure.