Scrape & Analyze Football Data with Kestra, Malloy and DuckDB

Accessing and analyzing football data has become more accessible with tools like Kestra, Malloy, and DuckDB, which streamline data scraping, processing, and analysis. Fbref.com offers a straightforward scraping process due to its user-friendly design and scraper policies, allowing data to be extracted and formatted into CSV files. Malloy, a new open-source language, simplifies data analysis by handling complex queries and data hierarchies while retaining SQL-like features, making it easier to compose reusable and nested computations. The orchestration tool Kestra integrates these processes by scheduling data scraping and analysis workflows using YAML files, enabling automated and seamless data management. This combination exemplifies a modern data stack, where domain-specific languages like Malloy and declarative orchestrators like Kestra enhance user experience by allowing fluid, semantic expression of data needs.