October 2017 Summaries

Astronomer, formerly known as UserCycle, evolved from a user analytics platform into a comprehensive data engineering hub by addressing the challenge of slow data integration faced by its clients. Initially focused on simplifying user behavioral analytics, the company expanded its infrastructure to include Apache Airflow® for workflow management and later considered technologies like Kafka for streaming infrastructure. Astronomer's platform now offers modular tools for data ingestion, such as Clickstream, and aims to provide agile, adaptable solutions that cater to companies seeking rapid innovation in analytics and data science. With a core API written in GraphQL, Astronomer emphasizes flexibility and customer-driven development, positioning itself as an alternative to long-term consulting projects by enabling swift, iterative progress in data management and analytics.

Founded by five NYU graduates, the nonprofit Advanced Research in Government Operations (ARGO Labs) aims to revolutionize data usage in local government by transitioning from a reactive "Excel for everything" mindset to proactive operations using Apache Airflow®. Emphasizing collaboration and code-based data infrastructure, ARGO leverages Airflow's open-source nature, allowing municipalities to maintain and extend data pipelines independently. In a notable initiative, ARGO organized a coalition of Californian water utilities to standardize disparate water data, using Airflow to ingest, parse, and refine the information for analytics, ultimately aiding in water management and conservation efforts. This approach has led to significant cost savings and improved planning for water utilities. Beyond water management, ARGO has expanded its efforts to include street quality assessments through projects like SQUID-Bike, which employs data collection and computer vision to evaluate urban infrastructure. By facilitating a shared data infrastructure, ARGO intends to prepare public services for the future, with a focus on transparency, trust, and efficient data management.