Monitor Databricks with Grafana Cloud for instant visibility into your workloads

Grafana Cloud offers a Databricks integration designed to provide comprehensive visibility into Databricks workloads, catering to the distinct needs of FinOps, platform, SRE, and analytics teams. This integration streamlines monitoring by importing metrics from Databricks workspaces directly into Grafana Cloud, eliminating the need for custom exporters or building dashboards from scratch. It includes three prebuilt dashboards that offer insights into costs, job reliability, and SQL warehouse performance, along with 14 alerting rules aimed at notifying relevant teams about significant changes or issues. The integration utilizes an open-source exporter, databricks-prometheus-exporter, which accesses Databricks System Tables for billing, job, and query performance data. Set up requires a Grafana Cloud account, Databricks credentials, and configuration through a setup wizard, with considerations for billing data lag and specific table permissions. The service aims to simplify the monitoring process and is part of Grafana Cloud's broader suite of tools for managing metrics, logs, and traces, with a free tier available for new users.