December 2024 Summaries

ASOS, a UK-based online retailer, has effectively enhanced its digital shopping experience and reduced Mean Time to Recovery (MTTR) by utilizing Grafana Cloud for improved observability. Initially struggling with scattered data and manual incident management, ASOS began using Grafana OSS in 2017 due to its "big tent" philosophy, which unifies diverse data sources into a single dashboard. This transition allowed ASOS to gain a comprehensive view of system performance from a user perspective. By 2022, ASOS fully migrated to Grafana Cloud, leveraging its enterprise plugins and integrations to streamline maintenance and focus on scaling its observability strategy. This move enabled ASOS to deprecate custom plugins, thereby reducing overhead and enhancing system reliability. The centralized monitoring provided by Grafana Cloud allows ASOS engineers to quickly identify and resolve issues, boosting confidence in their metrics and systems. As a result, the team is more prepared and confident in understanding and managing the metrics that matter most to their operations.

FalconX, a global crypto prime brokerage, transitioned to Grafana Cloud to enhance their observability infrastructure and achieve significant cost savings. Under the leadership of DevOps Manager Vaibhav Krishna, the company moved away from its expensive proprietary setup to adopt Grafana's open-source platform, which aligns with their ethos of using open standards and offers scalability, security, and compliance support. The migration to Grafana Cloud, completed in just a month, resulted in immediate cost reductions of 40%, primarily by optimizing log management through Grafana Loki, which is more cost-effective than OpenSearch. Despite initial skepticism about certain features like Adaptive Metrics, which significantly reduced unused time series metrics, the team was impressed by the platform's comprehensive functionality, including capabilities for metrics, logs, traces, and alerting that simplify observability processes and reduce administrative burdens. The centralized approach of Grafana Cloud helps streamline the management of their tech stack, making it easier for leaner teams to handle observability without the complexity of multiple tools, ultimately enhancing efficiency and focus.

Grafana is an open-source platform that transforms raw data into visually appealing dashboards, useful for a variety of purposes such as monitoring server health or local weather. This blog post serves as a guide for beginners to create a free weather dashboard using Grafana Cloud, illustrating how to convert raw JSON data from public APIs like the U.S. National Weather Service into an accessible format that can be viewed on mobile devices. Grafana Cloud, part of the Grafana LGTM Stack, supports a wide array of data sources and uses the Infinity data source plugin for ingesting data, allowing users to query and display information from various formats like JSON, CSV, XML, and GraphQL. The tutorial highlights steps such as signing up for Grafana Cloud, installing the Infinity plugin, and importing a preconfigured dashboard to customize with local weather data, demonstrating the ease of building comprehensive dashboards for personal or professional use.

OpenTelemetry, a framework focused on cloud-native systems, enables the standardization and transport of telemetry data across various applications, libraries, and languages without vendor lock-in, thus facilitating better insights through correlated metrics, logs, and traces. Discussed in an episode of "Grafana's Big Tent" podcast, the conversation shed light on the framework’s evolution and the importance of context in observability, highlighting how OpenTelemetry integrates with existing successful tools while addressing challenges like cost and the need for a cultural shift in the way distributed systems are operated. The podcast also touched on the governance of OpenTelemetry, emphasizing the collaborative efforts of its community, and hinted at future developments like the Entity SIG refactoring and profiling as a new signal, which could enhance debugging practices. Despite the complexities and trade-offs involved in its adoption, OpenTelemetry is seen as a critical tool for modern observability, bridging gaps between different telemetry data types to provide comprehensive system insights.

Grafana Labs fosters a culture of continuous learning by providing employees with various opportunities for professional development and personal growth. This includes a comprehensive onboarding process, flexible and autonomous learning options through their learning management system, Docebo, and access to LinkedIn Learning courses. The company also offers live learning sessions through their Concierge Services program, allowing employees to connect and learn in a group setting, either in person or virtually. Grafana Labs supports employees' individual learning journeys by providing a professional development budget of $1500 USD annually, which can be used for certifications, conferences, books, and more. This commitment to learning ensures that employees, known as Grafanistas, are continually expanding their knowledge and contributing to a thriving work environment.

Grafana Labs has updated its LLM plugin to provide users with greater flexibility in selecting large language models (LLMs) and providers that best suit their needs, emphasizing its commitment to a "big tent" philosophy. With the 0.10.0 release, the plugin now supports open-weights models and non-OpenAI providers, allowing users to integrate self-hosted or alternative LLMs via OpenAI-compatible APIs. This update enhances Grafana's generative AI capabilities, enabling features such as annotating dashboards, generating incident summaries, and explaining log patterns. The plugin's expanded support for different LLMs is aimed at balancing performance, privacy, and cost, and it includes options for deploying both base and large models depending on user requirements. Benchmarking tests have been conducted to ensure compatibility and performance across diverse LLMs, revealing that larger models generally offer better outcomes. Looking forward, Grafana aims to further integrate LLM functionalities, such as function calling and tool integration, to enhance user interaction with Grafana through various platforms and clients.

Umesh Pawar, a Senior Cloud Engineer, outlines a secure method for connecting Grafana to Google BigQuery using Workload Identity Federation, particularly when Grafana is running on Google Kubernetes Engine (GKE). This approach eliminates the need to expose a service account key, enhancing security while enabling the querying and visualization of BigQuery data directly in Grafana for varied use cases like log analysis and digital marketing campaign tracking. The process involves configuring Workload Identity Federation at the cluster level in GKE, creating IAM policies, and setting up a Kubernetes service account to allow Grafana to query the BigQuery API. The BigQuery plugin for Grafana is installed via the Grafana interface, where it can be verified and used to query data using SQL, benefiting from features like macros and autocompletion. This guide emphasizes the importance of granting precise permissions following the principle of least privilege and provides resources for further exploration of Workload Identity Federation and the BigQuery plugin.

In 2025, Grafana Labs anticipates significant advancements in observability, driven by trends such as the convergence of profiling and tracing, the strategic use of AI/ML, and the emerging role of eBPF in platform engineering. Profiling, despite being relatively new, is gaining traction alongside tracing to provide comprehensive insights into application performance. AI and machine learning, although not the ultimate solution, are expected to enhance engineers' capabilities by automating tasks and refining root cause analysis. While cloud repatriation is discussed as a cost-saving measure, most organizations will benefit more from optimizing their existing cloud infrastructure. eBPF is poised to revolutionize platform engineering by enabling seamless integration of system-level data and application telemetry. Open source projects like OpenTelemetry and Prometheus continue to dominate the observability landscape, offering vendor-neutral solutions and fostering innovation. These developments highlight a shift towards more integrated, efficient, and cost-effective observability practices for the future.

In 2024, Grafana Labs celebrated significant milestones, including the first in-person GrafanaCON in five years, held in Amsterdam, where they unveiled Grafana 11 and Loki 3.0. The year marked the introduction of Grafana Alloy, an open-source collector, and advancements in Adaptive Telemetry, which aims to streamline observability data using AI/ML. The company also hosted 10 hackathons, fostering innovation across teams, and launched the second season of their "Grafana's Big Tent" podcast. Grafana Labs extended its cloud offerings with new features to enhance observability and troubleshooting, earning recognition in industry rankings such as the Gartner Magic Quadrant and Forbes Cloud 100. They conducted their second annual Observability Survey, revealing insights into industry trends and challenges, and initiated the Grafana Labs Startup Program, providing substantial cloud credits to startups. Throughout these achievements, the company emphasized its commitment to the open-source community and its role in the evolving observability landscape.

PromQL vector matching is a crucial concept in Prometheus queries, which often involves handling vectors—a collection of time series data points that can be instant or range vectors, and may include labels. Dawid Dębowski, a software engineer, explains how arithmetic operations between vectors can result in unexpected outcomes, such as receiving a "No data" response, due to mismatches in vector labels. He illustrates this with an example of calculating the percentage of water-type Pokémons caught, highlighting that Prometheus compares labels on vectors for operations, dropping unmatched vectors. Solutions to ensure accurate results include using the same labels on both vectors or leveraging PromQL keywords like on and ignoring to guide Prometheus in matching vectors with different label sets. Additionally, the post touches on advanced techniques such as group_left and group_right for handling many-to-one vector matching scenarios, emphasizing the importance of understanding vector matching for accurate query results in real-world applications.

In 2024, Grafana Labs continued to prioritize open source development, marked by significant updates and new releases across its suite of tools. The company made strides in enhancing interoperability and user experience, with major updates to the Grafana LGTM Stack, including improvements to Loki, Grafana, Tempo, and Mimir. Grafana Loki introduced Bloom filters and native OpenTelemetry support, while Grafana saw enhancements in custom visualizations and alert management. Tempo introduced TraceQL metrics and improved performance, and Mimir focused on optimizing query execution and storage. The introduction of Grafana Alloy highlighted Grafana Labs' commitment to advancing OpenTelemetry integration, further showcased by their active participation in the OpenTelemetry community. The company also emphasized ease of use with the Explore apps suite for simplified data analysis and continued to support Prometheus and OpenTelemetry collaborations, reinforcing Grafana Labs' role as a leader in the open source observability landscape.

GrafanaCON 2025 is scheduled to take place in Seattle from May 6-8, offering a three-day event filled with technical sessions, live demos, user stories, and networking opportunities centered around Grafana and its open-source ecosystem, including Prometheus, OpenTelemetry, Loki, Mimir, and Tempo. Registration is open with early access available for those interested in attending, and there is a call for presentations from open-source contributors and practitioners to share their insights and experiences. The event will also feature the Golden Grot Awards, celebrating outstanding Grafana dashboards in personal and professional categories, with winners receiving a trip to Seattle to be honored on stage. As the date approaches, the community is encouraged to participate by submitting talks and dashboards, marking this as a key gathering for users and developers in the Grafana community.

Throughout 2024, Grafana Cloud made significant strides in enhancing its observability platform, focusing on efficiency, intelligence, and user accessibility. Key updates included the introduction of Adaptive Logs, building on the success of Adaptive Metrics to reduce observability costs by optimizing log data management. The Explore apps suite was expanded to simplify data analysis with user-friendly interfaces, enabling both beginners and experts to leverage telemetry data effectively. Grafana Cloud also enhanced its Synthetic Monitoring and Application Observability features, with new tools for complex transaction simulation and application performance insights. To manage observability workloads at scale, Fleet Management was introduced, providing efficient control over collectors. AI/ML advancements were notable, including contextualized root cause analysis workflows and AI Observability for generative AI applications, designed to optimize and diagnose AI systems. Infrastructure improvements focused on Kubernetes and multi-cloud monitoring, offering comprehensive insights across cloud services. Additionally, enhancements in Grafana SLO and new Enterprise data sources bolstered the platform's capabilities, making it easier to manage service level objectives and integrate external data systems. Overall, these developments underscore Grafana Cloud's commitment to advancing observability solutions across various domains.

Grafana's versatility in creating dashboards is showcased in various innovative projects from 2024, with applications spanning both personal and professional realms. At GrafanaCON 2024, the Golden Grot Awards spotlighted exceptional dashboards, such as Ruben Fernandez's personal dashboard that utilizes Grafana Cloud and Prometheus to streamline his morning commute in Atlanta, and Christopher Field's professional dashboard that revolutionizes defect analysis in steel alloys for nuclear reactors. The article highlights diverse uses of Grafana, from monitoring children's chores and visualizing home automation systems to identifying bird songs and tracking lunar landings by JAXA. Grafana's adaptability is further emphasized by its adoption in global industries, including the Danish company CSS Electronics' use of Grafana for visualizing CAN bus data and Schwarz IT's deployment of Grafana Enterprise across multiple organizations. Community-driven projects also illustrate Grafana's impact, such as tracking a diabetic child's health and assessing the energy efficiency of an induction cooktop, demonstrating the tool's broad applicability and potential for creative problem-solving.

In a recent episode of "Grafana's Big Tent" podcast, Grafana Labs CTO Tom Wilkie, alongside experts Thomas Dullien, Bryan Boreham, and Niki Manoledaki, delve into the intricacies of resource optimization and carbon footprint reduction in cloud computing. The discussion highlights the use of tools like Kepler and Karpenter for fleet-wide profiling and Kubernetes optimization, aiming to enhance efficiency and sustainability. Bryan Boreham shares insights on reducing Prometheus' CPU memory usage through meticulous profiling, which revealed that significant memory was consumed by metadata. The conversation also touches on the challenges of measuring carbon impact, with Thomas Dullien advocating for empirical metrics and collaboration with cloud providers to better estimate energy consumption. The episode underscores the importance of continuous profiling to uncover inefficiencies, likened to solving "murder mysteries," and emphasizes that while optimization can occur at various levels, the workload level remains critical for substantial cost reductions and resource optimization in cloud environments.

Grafana 11.4 introduces new capabilities for AWS users by integrating support for OpenSearch Piped Processing Language (PPL) and OpenSearch SQL in the AWS CloudWatch data source plugin, allowing users to choose their preferred query language for filtering and aggregating CloudWatch logs. This update enhances the flexibility of log querying and supports features such as syntax highlighting, live code completion, and sample queries to improve user experience. The release aims to enrich the Grafana ecosystem by inviting community engagement and feedback, offering a "forever-free" tier on Grafana Cloud, and providing resources for upgrading to the latest version. Grafana Labs expresses gratitude to its community for their contributions and encourages participation in the annual Observability Survey to further develop the platform.

Grafana Alerting has evolved significantly since its revamp in Grafana 9, with enhancements aimed at making alerts more robust and manageable. A key development is the introduction of Grafana-managed recording rules in Grafana 11.3, which streamline the process of setting up and managing incident responses by adopting concepts from the Prometheus alerting model. These recording rules precompute and store results of frequently needed or computationally expensive queries in a Prometheus time-series database, reducing the need for repetitive queries and thereby improving dashboard performance and reducing cloud costs. Additionally, they allow for integration and combination of data from multiple sources, facilitate experimentation with Prometheus without a full migration, and offer an open-source alternative to the enterprise recorded query feature, integrating seamlessly with Grafana's existing alerting infrastructure. Future enhancements for recording rules include configurable target data sources per rule and sequential evaluation of rules, aiming to further align with Prometheus and enhance consistency. Users are encouraged to explore the Grafana Alerting documentation for more information and provide feedback through Grafana's community channels.

Grafana Cloud has significantly enhanced its Synthetic Monitoring capabilities in 2024, focusing on making monitoring processes more efficient and impactful by integrating Grafana k6 into the architecture, allowing for more complex user journey emulations. The introduction of multiHTTP and k6 scripted checks has expanded testing capabilities beyond simple HTTP endpoint checks, enabling verification of multi-step API interactions and data flows within applications. Additionally, the UI has been streamlined to simplify the setup and understanding of running checks, while the pricing model has shifted from telemetry-based to execution-based, aligning with industry standards and providing customers with 100,000 monthly executions under the free tier. Existing customers are encouraged to optimize their Synthetic Monitoring configurations by reviewing usage, reducing execution frequency, and using fewer probe locations to manage costs effectively.

Grafana Cloud has introduced a new feature that allows users to query private network data securely using AWS PrivateLink and Amazon VPC Lattice, without deploying the Private Data Source Connect (PDC) agent within their AWS environment. This approach, which was in private preview as of December 2024, ensures that data flows through a secure, private network, minimizing exposure to external threats and adhering to compliance standards like GDPR and HIPAA. By utilizing AWS PrivateLink, users benefit from low-latency pathways, optimizing the performance of Grafana dashboards and maintaining cost efficiency by keeping transactions within the AWS network. This feature is particularly beneficial for industries where data security and integrity are paramount, such as financial services, healthcare, and technology, as it simplifies the integration process by eliminating complex VPN configurations and reducing the risk of data interception. Although the private preview for this feature ended in August 2025, users are encouraged to adopt the PDC agent-based approach for secure data querying.