Gorgias — Applying Infrastructure As Code Best Practice to Data Engineering with Kestra

Gorgias employs Infrastructure as Code (IaC) principles to enhance its data engineering practices using various tools like Kestra, Airbyte, dbt, Hightouch, and Terraform. By adopting a declarative approach, Kestra transforms data pipeline management, offering a seamless integration with Gorgias' existing toolkit, which includes BigQuery, PostgreSQL, and GitHub. This method simplifies task creation through an intuitive UI and supports native Webhook triggers, allowing modularity and versioning in workflows. Gorgias has shifted to a dedicated data orchestration tool to address the limitations of embedded schedulers, maintaining complex logic across systems with a centralized platform for monitoring and debugging. Terraform modules and Kestra subflows help abstract complexities for users, facilitating modular development and efficient configuration management. Kestra's integration as a CI/CD engine further supports Gorgias' infrastructure management, enabling streamlined operations and enhanced collaboration among data practitioners.