How ngrok uses Dagster to run our data platform

Ngrok utilizes Dagster to orchestrate its comprehensive data stack, which includes Apache Iceberg for data storage and AWS Athena for query processing, among other tools. This orchestration aids in managing a monthly influx of about 20TB of data across over 400 Dagster assets and around 170 distinct dbt models. Dagster is particularly valued for its approach to viewing data as assets and its adherence to software engineering standards in data pipelines, which simplifies the management of data lineage and enhances monitoring. The integration of Dagster with dbt allows ngrok to maintain SQL models in a structured manner, facilitating the visualization of data lineage and the automation of onboarding SQL transformations into dbt models. By deploying Dagster on their Kubernetes infrastructure using their ngrok Ingress Controller, ngrok ensures secure access and efficient management of both production and development instances, underscoring the benefits of combining Dagster's orchestration capabilities with ngrok's ingress-as-a-service platform.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.