Big Data London 2025: Notes From the Floor

Big Data London 2025 highlighted the ongoing fragmentation within the data industry, where practitioners operate across various layers of the technology stack, leading to a critical need for effective orchestration solutions. Despite the prominence of AI in discussions, its practical integration into workflows remains limited, with many vendors only offering surface-level features. The event underscored the importance of open-source collaboration, as it fosters meaningful exchanges and advances in integration, contrasting with the often sales-driven focus of closed-source vendors. Conversations frequently revolved around fundamental themes such as ETL processes and the use of Python, indicating that while business logic is inherently complex, orchestration should remain straightforward and declarative, best served by platforms like Kestra. The conference confirmed the necessity for a central platform that separates orchestration logic from business logic, simplifying the management of workflows in a fragmented data landscape.