January 2025 Summaries

Integrating performance testing and continuous profiling provides deeper insights into application behavior, allowing organizations to pinpoint issues like CPU bottlenecks and memory leaks during testing, rather than in production. This approach helps identify inefficiencies early by shifting left, integrating with CI/CD pipelines, ensuring applications perform optimally under various loads. The integration of Grafana Cloud k6 and Grafana Cloud Profiles offers a seamless way to connect performance tests with profiling data, providing a comprehensive view of code performance and real-world impact. This method not only enhances user experience by maintaining application speed and reliability during peak conditions but also helps in budget efficiency by addressing issues early in the development cycle. By applying middleware and annotating profiling data with performance test labels, teams can visualize and analyze performance metrics in the Grafana Cloud UI, allowing for proactive optimization before critical events like Black Friday. This integration ultimately transforms passing tests into delivering exceptional performance, making Grafana Cloud an accessible platform for continuous profiling, metrics, logs, traces, and dashboards.

Grafana 11.5 introduces new features aimed at enhancing data management, sharing, and security within the platform. The release includes a redesigned sharing experience for dashboards and panels, offering options for sharing links, embeds, and snapshots, while allowing customization of panel images. The update also streamlines the creation of ad-hoc filters and introduces the Plugin Frontend Sandbox to improve security by isolating plugin code. Additionally, the Elasticsearch data source plugin now supports cross-cluster search, simplifying data querying across multiple clusters. Migration to Grafana Cloud is made easier with the new Migration Assistant, which facilitates moving Grafana instances and plugins to the cloud. Furthermore, Private Data Source Connect support is expanded for AWS data sources, and Role-Based Access Control (RBAC) updates enhance management of notification policies and alerting. The release is complemented by community engagement and documentation resources to aid users in leveraging these updates effectively.

Organizations are increasingly migrating to Grafana IRM to enhance production reliability, reduce costs, and streamline their incident response tools. Grafana IRM offers observability-native workflows, tool consolidation, and cost-effective pricing, making it an attractive choice for many. The migration process can follow three main paths: hiring professional services for a seamless transition, using Grafana's IRM migrator tools for in-house management, or opting for self-migration without these tools, which can be executed via Terraform scripts or a UI-based approach. Each path caters to different organizational needs, such as resource availability, timelines, and technical expertise, and it is crucial to evaluate these factors to ensure a smooth migration. By selecting the appropriate migration strategy, organizations can fully leverage Grafana IRM's capabilities, achieving improved incident management and operational efficiency.

Grafana has become a crucial tool for industrial automation and the Industrial Internet of Things (IIoT), offering real-time data visualization and alerting capabilities across various sectors. It provides a centralized view of diverse data sets without the need to combine them in a backend store, making it particularly valuable in manufacturing environments where data originates from numerous systems and sensors. Grafana's integration into platforms like the United Manufacturing Hub's open-source IT/OT Integration Platform, and its use by companies such as Comact, Canon, and Siemens, highlights its adaptability and effectiveness in enhancing operational insights and reporting. Additionally, Grafana Cloud offers scalability for managing large data volumes, as demonstrated by its deployment at Dexory to monitor warehouse operations with autonomous robots. The ease of use and the extensive open-source community surrounding Grafana further contribute to its widespread adoption, enabling companies to create impactful visualizations with minimal barriers to entry.

Wilfried Roset, an engineering manager and Grafana Champion, emphasizes the importance of applying Service Level Objectives (SLOs) to databases using synthetic monitoring to enhance user experience and operational efficiency. SLOs, which are quantifiable targets for IT service performance and reliability, were introduced by Google and help maintain service quality by encompassing metrics like uptime, response times, and error rates. By viewing databases as APIs similar to HTTP services, SLOs can be effectively applied to databases, focusing on error rates and response times to improve service delivery. Real-traffic sampling and synthetic monitoring are two methods for implementing SLOs, with synthetic monitoring offering more control and independence from client-side metrics. Tools like sql_exporter and Grafana Mimir are highlighted for their roles in monitoring and tracking SLOs. Roset shares a success story where synthetic monitoring detected a network issue affecting database performance, underscoring the practicality and efficiency of SLOs in maintaining high-quality database services.

The Kubernetes Monitoring Helm chart 2.0 introduces significant updates aimed at simplifying and enhancing the user experience in collecting observability data from Kubernetes clusters. This new version allows users to easily gather metrics, logs, traces, and profiles using Grafana Alloy, an open-source distribution of the OpenTelemetry Collector, along with other utilities like Node Exporter and OpenCost. Version 2.0 emphasizes user-focused feature design, eliminating the need for users to understand underlying technologies; it also offers flexible options for defining multiple telemetry data destinations. Built-in integrations are now available for popular services, facilitating easier configuration and upgrades, while Fleet Management compatibility allows for scalable collector deployment management via Grafana Cloud. Migration from version 1.x is made straightforward with a migration guide and a utility for converting configuration files, and the developers encourage user feedback to drive further improvements.

Grafana Cloud has introduced several updates to enhance its fully managed observability platform, Grafana LGTM Stack, featuring Loki for logs, Grafana for visualization, Tempo for traces, and Mimir for metrics. This month's updates include enhancements to Grafana Cloud k6, offering a new API and a recommendations tool to streamline performance testing with tailored suggestions and improved integration with CI/CD processes. The platform also introduced customizable data visualization options and a new UI for machine learning metric forecasting, making it easier to set up forecasts and alerts. The Adaptive Logs feature now provides per-service recommendations to optimize log management and reduce observability costs. Additionally, Grafana SLO supports more flexible, Grafana-managed alert rules, and incident management tools like Grafana OnCall and Grafana Incident have been updated with service account token authentication and more flexible labeling systems. Lastly, updates to data source plugins include enhanced support for Elasticsearch's cross-cluster search and new features for the Datadog plugin, demonstrating Grafana Cloud's commitment to improving user experience and efficiency.

The blog post delves into the functionality and application of the OpenTelemetry Operator, which facilitates auto-instrumentation of Kubernetes applications without requiring code changes. By leveraging Kubernetes operators and custom resources, the tool enables seamless collection and processing of telemetry data, such as traces, metrics, and logs, by auto-injecting necessary agents into application pods. It elaborates on the installation process using tools like kind and cert-manager, and highlights the setup of the OpenTelemetry Collector and custom resources for different programming languages, such as Java, Python, .NET, and NodeJS, to handle auto-instrumentation. The post emphasizes the importance of annotations in deployment manifests to guide the operator on which applications to instrument, and concludes by suggesting Grafana Cloud as a platform for visualizing the telemetry data, which can enhance application observability with minimal setup.

Grafana's community thrives on members sharing their experiences and best practices, resulting in a wide range of topics that have gained popularity over the past year. Key areas of interest include visualizations, dashboards, and panels, which are essential for data representation in Grafana, as well as alerting, performance testing, and troubleshooting techniques. The community also focuses on utilizing Grafana Loki for log aggregation, Grafana Mimir for Prometheus metrics, and Grafana Cloud for managed observability solutions. Other popular subjects are Grafana Tempo for distributed tracing, Grafana Alloy for OpenTelemetry integration, and beginner content for new users. The community encourages the submission of content to further knowledge sharing and invites participation in GrafanaCON 2025, emphasizing the importance of community engagement in fostering growth and innovation within the Grafana ecosystem.

Grafana Tempo 2.7 introduces several performance and operational enhancements with additional features in TraceQL, including new metrics functions such as avg_over_time, min_over_time, and max_over_time, which provide more granular insights into trace data. The update focuses on reducing resource consumption in large Tempo clusters by optimizing query frontend resources and ingester memory usage. Minor breaking changes include updates to OpenTelemetry configurations for improved security and the adoption of Prometheus' fast regular expression matcher for enhanced performance and consistency across Grafana products. Future developments involve the RF1 rearchitecture to further improve TraceQL performance and reduce total cost of ownership, with community engagement encouraged through forums and community calls. The release also highlights the availability of Grafana Cloud's free tier, which offers users access to 50GB of traces, logs, and metrics.

Grafana Play, a platform created by Grafana's co-founder Torkel Ödegaard in 2015, has become a key resource for demonstrating Grafana's capabilities, allowing users to experiment with features and learn through sample dashboards. Originally designed as an open environment for creating and editing dashboards, it has seen significant growth and community engagement, with many users contributing their expertise. Recent updates focus on enhancing privacy and security by implementing a policy to delete inactive accounts that haven't logged in for over a year and haven't created dashboards, in line with data minimization principles. Most users interact with Grafana Play anonymously, so these changes will primarily impact inactive account holders, while active contributors and users will remain unaffected. The platform continues to encourage community contributions, offering a dedicated Slack channel for discussions and feedback, reinforcing its role as a valuable educational and collaborative resource.

Adaptive Telemetry is a feature within Grafana Cloud designed to enhance the efficiency of observability data management by reducing costs, noise, and mean time to recovery (MTTR) while preserving valuable telemetry. It addresses the issues arising from the traditional "collect everything" approach, which often leads to data overload, increased costs, and difficulty in identifying critical signals. By automating the classification and prioritization of signals through tools such as Adaptive Metrics and Adaptive Logs, Adaptive Telemetry helps organizations focus on actionable data, reducing unnecessary storage and processing expenses. It offers flexibility, allowing users to apply exemptions for critical data while continuously learning and adapting to changing usage patterns. Real-world applications, such as those by TeleTracking, SailPoint, and Dell Technologies, demonstrate how Adaptive Telemetry can lead to significant cost reductions and improved team productivity. The feature is set to expand to include traces and profiles, ensuring that every piece of telemetry data stored is valuable, reflecting Grafana Cloud's mission to optimize data observability.

Grafana SLO is a tool designed to help software teams effectively set and manage service-level objectives (SLOs) by leveraging statistical predictions based on historical data. By using 90 days of data and simulations, it provides a visual representation of the risk associated with meeting specific targets, enabling teams to make informed decisions about their service levels. The application helps avoid arbitrary or unrealistic target-setting by providing insights into the probability of achieving goals, thereby aligning SLOs with customer satisfaction and business priorities. Grafana SLO aims to empower service owners to balance reliability with operational velocity, suggesting that understanding and quantifying risk can lead to better service agreements and customer retention strategies. As the feature matures, there is potential for enhancements such as more accurate risk modeling and improved alerting systems, ensuring that SLOs remain meaningful and aligned with user expectations.

Prusa Research, a company well-known for its 3D printers, collaborates with Grafana Labs to enhance the observability of its expansive printer farm using Grafana's tools. The integration of observability into 3D printing allows for the optimization of printer functions, such as testing heatbed probing and filament sensor values, which are crucial for improving printer reliability and efficiency. Prusa's use of open-source designs and their extensive printer farm, consisting of over 700 units, exemplifies their commitment to product reliability by testing and using their printers for production. Grafana's observability tools, particularly their transition from Thanos to Mimir for better data querying, plays a vital role in managing and monitoring the printer farm, highlighting the synergy between software observability and hardware innovation. The collaboration underscores a shared philosophy of using and testing products internally to ensure quality, much like Grafana Labs does with its software platforms.

ObservabilityCON on the Road is set to bring the highlights of the flagship open-source observability conference, ObservabilityCON 2024, to various cities in 2025, including the Bay Area, Tokyo, Chicago, and Sydney. These one-day events will replicate the core features of the original conference, such as technical deep dives, live demos, and expert Q&A sessions. Additionally, Grafana Labs experts will collaborate with local observability professionals in each city to share their success stories and insights. Registration is now open for these events, which aim to engage and extend the observability community by making the conference more accessible to those unable to attend the main event in New York City.

GrafanaCON 2025 invites community members to apply to speak at its annual conference in Seattle from May 6-8, emphasizing the open-source community's role at the event. The conference will offer unique spaces, hands-on labs, live demos, and sessions featuring Grafana and its ecosystem, including Prometheus, Loki, and OpenTelemetry. Organizers seek speakers to share real-world experiences, from unique use cases to innovative dashboards, and offer support in preparing presentations. Applications are open to contributors and practitioners from all backgrounds, with proposals for lightning talks and use case stories particularly encouraged. The event aims to highlight the diverse and creative ways Grafana is used, fostering a spirit of collaboration and knowledge exchange within the open-source community.

OpenTelemetry, a rapidly growing open-source project in the cloud-native ecosystem, experienced significant developments in 2024, including a 45% increase in code commits and a doubling in Google search volume, signaling its rising prominence. Major milestones included the stabilization of the profiling signal, which allows users to correlate resource telemetry data with code components, and the Spring Boot starter, simplifying the instrumentation of Spring Boot applications. Updates to Semantic Conventions and the release of new open-source code for the OpenTelemetry Datadog receiver further enhanced its capabilities. Grafana Labs, a leading contributor to both OpenTelemetry and Prometheus, integrated support for the project into its products, such as Grafana Beyla and Grafana Alloy, while also advancing application observability solutions. Looking ahead to 2025, the OpenTelemetry community aims to expand eBPF instrumentation, enhance language-level support, and stabilize the OpenTelemetry Collector, with Grafana Labs continuing to foster collaboration and innovation within the community.

The National Library of the Netherlands is dedicated to providing seamless access to both physical and digital literature, storing 3.5 petabytes of data that grows annually, and making it available online around the clock. To support this mission, they implemented the Grafana stack for their observability strategy, prioritizing user experience and real-time data-driven decision-making. The library chose Grafana over Splunk due to its ease of use, cost-effectiveness, and native Prometheus support, leading to an annual operational cost reduction of €80,000. They deployed two Grafana servers with Grafana Mimir for metrics and Grafana Loki for logs, enhancing developer empowerment and reducing mean time to recovery (MTTR) by enabling quick issue identification through customized dashboards. Looking forward, the library plans to expand its Grafana stack capabilities with components like Grafana Alerting, Tempo, and Faro for enhanced performance monitoring and user experience.

Grafana Alloy, developed by Grafana Labs, is an open-source telemetry collector designed to simplify and enhance observability at scale by integrating seamlessly with the OpenTelemetry and Prometheus ecosystems. As a continuation of Grafana Agent Flow, Alloy is OTLP-compatible and incorporates Prometheus optimizations, making it ideal for cloud-native infrastructure and application observability tasks. Users can utilize Alloy to process, transform, and send metrics, logs, and other telemetry data to Grafana Cloud, with built-in support for alerting rules and live debugging features. The platform supports sending OTLP metrics, Prometheus metrics, and logs using various components such as otelcol.receiver.otlp, prometheus.scrape, and loki.write, and offers comprehensive resources and demonstrations to assist users in setting up and managing their observability pipelines. With Grafana Agent and Grafana Agent Operator being deprecated, transitioning to Alloy is encouraged for continued support and functionality. Grafana Cloud provides a robust platform for managing these telemetry data with flexible pricing plans, including a free tier.

CERN, renowned for operating the world's largest particle accelerator, also manages the world's largest computing grid, the Worldwide Large Hadron Collider Computing Grid (WLCG), which spans 170 institutes across 42 countries and supports immense data processing needs. To monitor this vast infrastructure, CERN utilizes Grafana and Grafana Mimir, tools that provide crucial visualization and alerting functions, allowing the organization to handle diverse data sources efficiently. Grafana serves as CERN's primary monitoring interface, with over 5,000 users creating thousands of dashboards, while Grafana Mimir addresses the challenges of integrating long-term storage for Prometheus metrics, proving scalable and flexible for their extensive operations. CERN's deployment includes both public and private instances, ensuring accessibility and security, and Mimir's integration has enhanced their data handling capabilities, allowing for significant scaling and multi-tenancy. Looking ahead, CERN is considering additional open-source tools like Open Telemetry and Grafana Tempo to further enhance their monitoring capabilities, highlighting the foundational role Grafana plays in their infrastructure.