Using Apache Airflow® to Create Data Infrastructure in the Public Sector

Founded by five NYU graduates, the nonprofit Advanced Research in Government Operations (ARGO Labs) aims to revolutionize data usage in local government by transitioning from a reactive "Excel for everything" mindset to proactive operations using Apache Airflow®. Emphasizing collaboration and code-based data infrastructure, ARGO leverages Airflow's open-source nature, allowing municipalities to maintain and extend data pipelines independently. In a notable initiative, ARGO organized a coalition of Californian water utilities to standardize disparate water data, using Airflow to ingest, parse, and refine the information for analytics, ultimately aiding in water management and conservation efforts. This approach has led to significant cost savings and improved planning for water utilities. Beyond water management, ARGO has expanded its efforts to include street quality assessments through projects like SQUID-Bike, which employs data collection and computer vision to evaluate urban infrastructure. By facilitating a shared data infrastructure, ARGO intends to prepare public services for the future, with a focus on transparency, trust, and efficient data management.