February 2024 Summaries

Grafana Tempo 2.4 introduces several key features and enhancements aimed at boosting performance and operational capabilities, including the experimental TraceQL metrics feature, which allows for creating metrics from traces and supports adhoc aggregation using TraceQL queries. This release also brings tiered caching with a simplified configuration that improves cache management and monitoring, as well as TCO improvements by reducing unnecessary backend polling. Additionally, Tempo now supports cross-tenant queries, making it easier to run queries across multiple tenants in multi-tenant modes, and vParquet3 is set as the default storage backend. Future developments focus on expanding TraceQL metrics capabilities and gRPC streaming support, with ongoing efforts to enhance the TraceQL language. The release encourages community engagement through various platforms and invites users to explore opportunities at Grafana Labs, while also promoting the Grafana Cloud free tier for getting started with Tempo.

Grafana Labs has introduced several updates and features to enhance the capabilities of Grafana Cloud, focusing on improved incident response, expanded plugin offerings, and enhanced observability for Kubernetes and other systems. Notable advancements include Span Profiles, offering detailed tracing and profiling for better application performance analysis, and structured metadata in Cloud Logs to improve query efficiency. A new alerting experience in Kubernetes Monitoring simplifies incident management by providing a comprehensive view of alerts. The AI-powered Sift feature aids incident investigations by automating error detection and context gathering. Additionally, Grafana Cloud now supports new Enterprise plugins, such as those for Sumo Logic and PagerDuty, and has introduced multi-stack data sources to streamline querying across multiple tenants. These developments aim to optimize performance, reduce costs, and simplify monitoring and alerting processes for users, offering flexibility across various use cases and integrations.

The Golden Grot Awards have returned for their second year to honor outstanding Grafana dashboards across both professional and personal categories, with winners to be celebrated at GrafanaCON in Amsterdam. The competition, which received numerous submissions, has narrowed down to ten finalists through expert judging, and public voting is open until March 1. Highlighted personal dashboards include Ruben Fernandez's tool for optimizing his Atlanta commute using various data feeds, Andreas Rahimic Andersen's Counter-Strike stats tracker, Asep Supriyadi's BI dashboard for shipping analytics, Amar Bin Lokman's water quality monitor, and Josh Gitlin's household temperature tracker. Professional entries feature Christopher Field's tool for monitoring steel alloy defects in nuclear reactor environments, Satoshi Takano's cognitive test results dashboard for athletes, Varun Bhaaskar's application health assessment, Wilson Acero's volcanic earthquake monitor, and Martin Olsson's driver interface for the Candela P-12 electric hydrofoiling ferry. These dashboards illustrate the versatility and innovative uses of Grafana, inviting both creative and practical applications in diverse fields.

Brian LeBlanc, a senior solutions engineer at Grafana Labs, describes how he addressed the challenge of accessing his home network remotely while ensuring security by deploying a VPN and monitoring it with Grafana Cloud. He utilized Wireguard for its security features but enhanced it with Grafana Agent to capture and forward VPN access logs to Grafana Cloud Logs, enabling real-time alerts and notifications via Slack for any unauthorized access. This setup, leveraging the free tier of Grafana Cloud, allowed him to maintain security without relying on third-party key management. By integrating multiple Grafana Cloud services, LeBlanc not only secured his network but also gained a deeper understanding of Grafana's product suite, improving his ability to assist customers in his professional role.

Rocky Linux, a community-driven and open-source operating system backed by CIQ, offers a powerful alternative to Red Hat Enterprise Linux (RHEL) and fills the gap left by changes in the CentOS project. The Linux integration for Grafana Cloud provides an efficient way to monitor Rocky Linux by using the node_exporter embedded in Grafana Agent, enabling comprehensive monitoring of logs and metrics. Grafana Cloud's integration for Rocky Linux includes pre-built dashboards for various system metrics such as CPU, memory, and network, along with alerts for resource usage. Users can easily set up this monitoring by creating a Grafana Cloud account and following the configuration steps provided. CIQ enhances Rocky Linux's appeal by offering enterprise support, security, and automation tools, while Grafana Labs promotes a versatile "big tent" philosophy, aiming to provide consistent user experiences across various data sources and environments through its free-tier and scalable plans.

Caddy, an open-source web server project, recently transitioned to using Grafana Cloud for improved observability of its infrastructure, motivated by the need for a more comprehensive and proactive monitoring solution. The switch was driven by challenges in managing server health and performance, particularly with Caddy's buildworker node, which experienced issues such as goroutine leaks and high CPU utilization. By adopting Grafana Cloud's observability tools, Caddy's team was able to conduct faster root cause analysis and enhance their server's performance by identifying it as CPU-bound rather than memory-bound. This migration allows Caddy to better diagnose and address issues, leveraging features like log aggregation, continuous monitoring, profiling, and synthetic monitoring to enhance their overall system reliability and efficiency. The collaboration with Grafana Cloud not only resolved immediate infrastructure problems but also set the stage for ongoing improvements in managing the Caddy project's development and operations.

Sift is a machine-learning-powered diagnostic feature in Grafana Cloud designed to assist SREs and DevOps teams in automating various aspects of incident investigation, such as detecting new errors, identifying recent deployments, and recognizing overloaded Kubernetes nodes. Since its public preview launch, Sift has introduced several new features, including an improved homepage, expanded investigation capabilities beyond Kubernetes, and a new outgoing webhook in Grafana OnCall for automatic investigations. Users can now trigger Sift investigations directly from various Grafana pages, with options to configure checks and customize settings for more effective diagnostics. A notable addition is the HTTP Error Series check, which identifies increased HTTP error rates and provides visual metrics for quick problem identification. Sift's investigation timeline feature correlates events for easier analysis, and the platform continues to evolve with plans for enhanced label selection and new diagnostic checks. Available at no extra cost in all Grafana Cloud tiers, Sift aims to streamline incident response and reduce mean time to recovery (MTTR).

The blog post provides a guide on how to instrument a Python application using OpenTelemetry and visualize the data in Grafana Cloud's Application Observability. It outlines steps for creating a Python application, installing the OpenTelemetry Python package, configuring the application to send telemetry data to Grafana Cloud, and observing the application's performance through Grafana's interface. The guide emphasizes that while it uses Grafana Cloud, OpenTelemetry is vendor-neutral, allowing the process to be adapted to other backends that support the OTLP protocol. It also highlights the benefits of Grafana Cloud's Application Observability, such as its integration with Prometheus for storing and visualizing telemetry data, and its capabilities for monitoring applications and reducing mean time to resolution (MTTR) by providing tools like Service Inventory, Service Overview, and Service Map. The guide is intended for local development or evaluation setups, and it encourages users to customize the setup according to their own backend needs if not using Grafana Cloud.

OpenTelemetry is gaining traction as an open-source project for monitoring applications, prompting organizations to carefully choose vendors that support this technology. Key considerations when selecting a vendor include assessing their commitment to OpenTelemetry and open standards, understanding how their observability solutions manage OpenTelemetry data, and evaluating the features available for managing and optimizing costs. Vendors like Grafana Labs actively contribute to OpenTelemetry and focus on interoperability with other tools such as Prometheus, offering native support for OpenTelemetry protocol data across metrics, logs, and traces. They also provide tools for cost management and predictability, ensuring that users can efficiently collect and utilize the most pertinent data.

Grafana Cloud has introduced an enhanced Kubernetes Monitoring application designed to simplify anomaly detection and alert management in Kubernetes clusters, which addresses the complexity often faced by DevOps teams and SREs. The new alerting experience offers a comprehensive overview of clusters, allowing users to filter alerts by cluster, namespace, and severity, and navigate directly to issues for quick resolution. This update eliminates the need for context switching between Grafana Alerting and Kubernetes Monitoring, streamlining incident response through direct access to metrics and log data. It also provides recommended actions based on specific workload configurations, aiding both inexperienced and seasoned users in managing resources effectively. Additionally, new predictive features leveraging Grafana Machine Learning allow users to anticipate future resource consumption issues. Existing users can seamlessly transition to the new functionality, which maintains all previous alerting rules, while new users can access comprehensive documentation to get started with Kubernetes Monitoring in Grafana Cloud, which offers a free tier and flexible plans.

Grafana has released multiple patch versions, including 10.3.3, 10.2.4, 10.1.7, 10.0.11, and 9.5.16, to address a medium severity security vulnerability identified as CVE-2023-6152 in its basic authentication system. This vulnerability allows users to bypass email verification, potentially preventing legitimate email owners from signing up if an email address is changed post-registration or used as a login name without verification. Affected versions span from Grafana 10.3.0 to 10.3.1, 10.2.0 to 10.2.3, 10.1.0 to 10.1.6, 10.0.0 to 10.0.10, and all versions older than 9.5.16. Users are advised to upgrade to the patched versions or disable basic authentication if alternative mechanisms are available. The vulnerability was disclosed through a bug bounty program in November 2023, and a fix was implemented by January 2024, with the public release of patches occurring in February 2024. Grafana Labs encourages reporting security issues through their dedicated page and maintains a security blog for updates on vulnerabilities and fixes.

"The Story of Grafana" is a four-part documentary chronicling the journey of Grafana from its inception as a personal project by Torkel Ödegaard in 2013 to becoming a leading data visualization tool with over 20 million users. Directed by Collins Pace, the series delves into Grafana's community-driven growth, its evolution into a thriving open-source business, and its expansion into a comprehensive observability platform. Featuring interviews with Grafana Labs co-founders and community contributors, the documentary highlights the company's commitment to building technology for practitioners and its mission to sustain an open-source foundation while serving a global audience. The series also explores the development of the Grafana LGTM stack, which includes tools for logs, metrics, traces, and more, as it transitions to a broader observability solution with both managed and self-managed offerings.

Mat Ryer discusses his evolved approach to writing HTTP services in Go, building on insights from his previous popular blog post and years of experience with the Go Time podcast and community interactions. He emphasizes structuring servers and handlers for maintainability, optimizing startup and shutdown processes, and handling common request tasks effectively. The article details changes in his methodology, such as avoiding surprise dependencies by passing them as arguments and embracing type safety through longer argument lists. Ryer highlights the importance of properly testing services, using a simple main function to enhance code readability and maintainability, and optimizing for testability by calling the run function during tests. He also discusses the use of middleware, design patterns like the adapter pattern, and the sync.Once pattern to defer setup, stressing the value of designing for testability and end-to-end testing. Throughout, he shares examples and insights from his work at Grafana Labs, underscoring the utility of these practices in both small and large projects.

GrafanaCON 2024 is set to be a major gathering for the Grafana open-source community, taking place in Amsterdam on April 9-10, with registration now open despite the event being sold out; a waitlist and virtual attendance option are available. Attendees can expect a range of activities, including technical talks, demos, hands-on sessions, and networking opportunities, with highlights like the Grafana 11 feature showcase, OSS office hours, and an IoT monitoring lab. The event also encourages community participation through the call for speakers and the Golden Grot Awards, which recognize outstanding Grafana dashboards in professional and personal categories, offering winners a chance to be honored in Amsterdam. As the countdown to the event continues, session details will be released, and community members are invited to submit talks and dashboards for the awards, fostering a vibrant exchange of knowledge and innovation within the Grafana ecosystem.

Grafana k6 v0.49.0 introduces several new features, including a built-in web dashboard for real-time visualization of test results and the promotion of the gRPC streaming API to a core module. The release enhances the k6 browser module by generating traces for browser actions and adding new methods for interacting with input fields. In Grafana Cloud k6, static IPs have been introduced for enhanced security, alongside improvements in user-defined project limits and enhancements to the k6 datasource. The k6-operator v0.0.13 includes updates for better security in Kubernetes setups and improvements for the Helm chart. The k6 extensions ecosystem has seen updates and new additions, such as xk6-kubernetes v0.9.0 and new extensions for HTTP/1.1 and WebTransport protocol support. The release encourages users to engage with the community through various platforms and offers a free tier for performance testing on Grafana Cloud.

"The Story of Grafana" is a four-part documentary that chronicles the evolution of Grafana from its inception as a tool to visualize time series data to its current status as a key component of the open-source Grafana LGTM stack, used by over 20 million users globally. Created by Torkel Ödegaard, Grafana emerged beyond its initial IT and observability niche into a versatile visualization tool, forming the backbone of Grafana Cloud and Grafana Enterprise platforms, which integrate seamlessly with over 100 data sources and include various tools for incident management, continuous profiling, and performance testing. Highlighting its commitment to interoperability, the docuseries features interviews with Grafana Labs co-founders and engineers, emphasizing the significant role of the open-source community in its development and emphasizing the company's philosophy of community engagement and open-source business practices. The documentary provides insight into the company's growth and adaptation, showcasing Grafana's transition from a focus on open-source enthusiasts to a broader audience, supported by an expanding plugin ecosystem, which underscores its transformative impact on data visualization and observability.

Grafana Labs has introduced Span Profiles in Grafana 10.3, a feature that enhances application profiling by combining tracing and profiling for more detailed observability. Unlike traditional continuous profiling, which provides a broad overview over fixed intervals, Span Profiles offers dynamic, focused analysis on specific execution scopes, such as individual requests or trace spans, enabling engineering teams to identify and address performance bottlenecks more efficiently. This integration allows for seamless transitions from high-level trace overviews to in-depth analyses of specific trace spans, enriching trace data with actionable insights and leading to significant performance improvements. For instance, Grafana Labs utilized Span Profiles to improve CPU utilization by four times, reduce API calls to object storage by three times, and achieve substantial cost savings in their operations. The feature is available in Grafana Cloud and Grafana 10.3, providing a powerful tool for optimizing application performance and reducing operational costs.

Grafana Labs has taken over the Infinity data source plugin, originally a side project, which has become a popular solution for visualizing data from diverse formats such as JSON, CSV, XML, and GraphQL within the Grafana community. Initially designed to overcome limitations of other plugins, Infinity quickly gained traction due to its ability to handle complex data without middleware and its support for key Grafana features like alerting. The plugin's growth led to a vibrant community that contributed to its development and feature expansion, resulting in millions of downloads. Now under Grafana Labs' stewardship, the plugin will benefit from enhanced support, new feature integration, and official documentation, while remaining open source and continuing to serve as a recommended plugin for various data visualization needs.

Grafana Labs is participating in FOSDEM 2024, a major open source conference in Brussels, with over 50 team members attending to engage with the community through talks, volunteering, and networking. The event, which is free and volunteer-run, expects more than 8,000 attendees and features over 867 sessions. Grafana Labs will have a stand at the conference for the sixth year, showcasing their open source Grafana LGTM stack and interacting with visitors. They are also hosting the Monitoring and Observability devroom, with sessions on topics such as eBPF, CI/CD observability, and distributed traces. Grafana Labs' engineers will present across various platforms, including keynote and lightning talks, covering themes like codifying technical documentation and cultural commitment to documentation maintenance. Attendees are encouraged to visit the Grafana stand or engage with team members throughout the conference.

The Sumo Logic Enterprise plugin for Grafana enables users to seamlessly integrate Sumo Logic's cloud-based log management and analytics platform with the Grafana dashboard, allowing for a unified view of log data and telemetry from various data sources. It simplifies the process of visualizing and correlating log data, offering features such as annotations, alerting capabilities, and centralized management of logs and metrics, which enhance real-time monitoring and troubleshooting of modern cloud-native applications at scale. The plugin is part of a suite of over 20 Enterprise plugins available in Grafana Enterprise and Grafana Cloud, including a free tier, and is designed to democratize data access and derive contextual insights, making it easier for organizations to maintain observability and improve system performance and security.