December 2024 Summaries

The Data Platform Week 2024 event was hosted by Dagster, a company focused on building a unified control plane for data platforms. The event featured talks and showcases from experts in the field, highlighting the importance of orchestration, integrations, transformation, and system of record in modern data platforms. Key features announced during the event include Core Orchestration with Declarative Automation, Integrations Ecosystem, and Orchestration Unification Federation & Airlift. The event also included showcases from power users who shared their experiences with Dagster's tools, such as Declarative Automation and Pipes. The event aimed to address the challenges faced by data engineers in modern organizations, including legacy technologies, data quality, and artificial intelligence.

Interactive debugging with Dagster and Docker is made possible by leveraging the `debugpy` library to set up a remote debugging session in the code location container, while utilizing Docker Compose to orchestrate the containers. This setup allows for seamless debugging of assets, schedules, and other code within the containerized environment. The debug configuration in VSCode enables connection to the remote debugger running in the code location container, facilitating step-by-step debugging of the code. By using this approach, developers can ensure a smoother development experience and quickly find and fix bugs, making it an essential tool for software and data engineering.