October 2023 Summaries

The OpenTelemetry .NET Automatic Instrumentation simplifies the process of collecting telemetry data from .NET applications without requiring code changes, making it particularly useful for large codebases or for those new to OpenTelemetry. This method allows for seamless integration with Grafana Cloud by setting up a Grafana Agent to handle OpenTelemetry Protocol (OTLP) data, enabling the visualization of metrics, logs, and traces from .NET applications. The auto-instrumentation relies on startup hooks, profiler hooks, and bytecode manipulation, allowing for customizable configurations via environment variables. It provides a streamlined approach to emitting high-quality telemetry data with minimal effort, offering a practical alternative to manual instrumentation, which can be complex and time-consuming. The tool is especially advantageous for users seeking an easy way to monitor application performance without extensive instrumentation efforts, and it can be easily incorporated into existing deployment workflows. For more advanced configurations and scenarios, users are encouraged to refer to the official OpenTelemetry documentation. Grafana Cloud offers a user-friendly platform to start with metrics and telemetry visualization, featuring a free tier to accommodate various use cases.

The blog post explores the process of creating a logs application plugin using Grafana's plugin tools, Grafana Scenes, and Grafana Loki, an open-source log aggregation system. By leveraging Grafana Scenes, a new front-end library, developers can create dynamic, dashboard-like experiences within Grafana application plugins. The guide outlines the setup requirements, including Node.js, Yarn or npm, and Docker, and provides step-by-step instructions for creating a project, configuring a Loki data source, and composing a scene using Grafana Scenes. The post details how to build queries and visualizations, such as stat, time series, and logs panels, to effectively handle and display log data. It emphasizes the expressive capabilities of the Scenes library in constructing dynamic user interfaces and encourages readers to explore further resources to enhance their understanding and use of Grafana's tools. The blog concludes with a prompt to try Grafana Cloud, highlighting its free tier and various plans.

The integration of a Spring Boot application with Grafana using OpenTelemetry standards involves a series of steps designed to optimize application performance through effective monitoring and visualization. This process begins by setting up Java instrumentation for the Spring Boot app using OpenTelemetry, allowing for the collection of telemetry data such as metrics, logs, and traces. The guide then details the configuration of an OpenTelemetry Collector to receive, process, and export telemetry data to tools like Prometheus and Grafana Loki. Using Docker Compose, the application and the OpenTelemetry Collector are set up to communicate over a shared network, facilitating telemetry data exchange. For deployment, the guide suggests using Amazon ECS to manage the lifecycle of the Spring Boot app and the OpenTelemetry Collector as separate entities within a single ECS task, ensuring seamless communication and flexibility. Finally, the collected data can be utilized in Grafana to create dashboards and analyze application performance, thereby enabling developers to make informed optimization decisions.

KubeCon + CloudNativeCon North America 2023, taking place in Chicago from November 6-9, is a highly anticipated event for the cloud native community, featuring discussions on IT sustainability, Kubernetes cost monitoring, and advances in observability. Grafana Labs, a Gold sponsor, will be actively participating with demonstrations and talks, focusing on environmental sustainability in the cloud, performance insights using flamegraphs, and measuring Kubernetes costs. Key topics include power consumption accountability, energy efficiency strategies, and the integration of performance profiling into OpenTelemetry, with a special emphasis on the cost optimization techniques of Kubernetes clusters using OpenCost. The event will showcase leading experts from various organizations, presenting insights into tackling sustainability challenges and performance optimization across diverse application landscapes.

The new Grafana developer portal is designed as a centralized resource for developers aiming to enhance Grafana's capabilities through plugins and applications. This portal addresses the growing demand for a dedicated space where developers can access essential documentation, tutorials, and tools, thereby complementing Grafana's existing documentation. It offers a streamlined user experience with resources such as the create-plugin CLI tool, the Grafana Plugin SDK for Go, and guides on signing and publishing plugins. Additionally, the portal facilitates the creation of user interfaces via the React component library and provides guidance on using the Grafana Scenes API to build experiences similar to Grafana dashboards. Developers are encouraged to engage with the community, contribute to open-source projects, and provide feedback to improve the portal further. Under the leadership of Timur Tolzhabayev and David Harris, the launch of the portal is a significant step in enhancing the development experience, promising frequent updates and new content to keep developers informed about the latest in Grafana development.

Web performance testing with the Grafana k6 browser and Grafana Faro involves using both lab data and field data to get a comprehensive view of an application's performance. Lab data, collected in controlled environments, is useful for early detection and resolution of performance issues, while field data, gathered from real user interactions, reflects actual user experiences under varying conditions. By integrating these two types of data, organizations can better understand and improve their web applications. Grafana Faro, a Web SDK, helps collect field data, including performance metrics and errors, offering insights into real-world user experiences. The combination of Grafana k6 browser and Grafana Faro enhances visibility into web performance, enabling proactive monitoring and more reliable applications. This approach highlights the importance of continuous testing, where feedback from field data informs and refines lab testing, ultimately leading to more robust and user-friendly web applications.

Grafana 10.2 introduces a range of enhancements aimed at improving dashboard and data visualization capabilities, including AI-generated panel titles and descriptions, interactive visualization features, and streamlined alerting processes. The release includes significant updates to Grafana Tempo, enabling the direct display of RED metrics from aggregated traces, and introduces new tools for better user and service account management in Grafana Enterprise. Additionally, public dashboards can now be shared without requiring login, and there's improved integration with Grafana OnCall for alert management. Other features include the support of template variables in the Grafana Pyroscope data source, temporary AWS credentials for easier CloudWatch integration, and improved dashboard navigation and management. The release highlights Grafana's commitment to user feedback and continuous innovation, while offering a free tier through Grafana Cloud for new users to explore these features.

Antonio Calero Merello and Laura Fernández Fernández from Grafana Labs provide a detailed guide on monitoring temperature and humidity using Grafana, Raspberry Pi, and various sensors, with a focus on DHT22 and RuuviTag sensors. They explain the process of setting up a Raspberry Pi with a Python application to collect sensor data, which is then containerized using Docker and exposed via an HTTP endpoint for Prometheus to scrape and store in a time-series database. Grafana is utilized to create real-time dashboards for data visualization. The blog outlines the steps to connect sensors to the Raspberry Pi, configure the Grafana stack, and use Prometheus to manage data collection. Additionally, the guide highlights the importance of monitoring environmental conditions for various applications and offers insights into using Grafana's alerting system to receive notifications based on sensor data.

Flexcity, a leader in electric flexibility and a subsidiary of Veolia, played a critical role in stabilizing the French national power grid during a severe energy crisis exacerbated by the Russian invasion of Ukraine and a nuclear power outage in France. The company was tasked by the French grid operator, RTE, to balance electricity supply and demand, reducing the risk of shortages. Flexcity transitioned from a homegrown monitoring tool to Grafana Cloud, enabling real-time tracking of energy consumption through dashboards that displayed customers' power usage every four seconds. This switch allowed Flexcity to efficiently manage demand response, where devices adjust their power consumption based on grid needs, ensuring stability during peak periods. The move to Grafana Cloud provided Flexcity with enhanced features such as managed security updates, single sign-on, white-label dashboards for customer branding, and responsive customer support. These improvements contributed to a more confident and less stressful operation during winter, a crucial season for grid activations.

Grafana k6 has released version 0.47.0, offering new features like gRPC's binary and reflection metadata support, enhanced Trend metrics precision in Grafana Cloud, and ARM64 Docker images. The update includes new authentication methods for Prometheus remote write output and improvements in the k6 browser, such as a stabilized Cookie API and Docker support for browser-based tests. Grafana Cloud k6 continues to expand with features like Distributed Tracing, browser testing, and an Analysis tab for custom metrics. The k6-operator has introduced a new pre-release candidate with Helm chart deployment options and a new TestRun resource. Additionally, the k6 extensions ecosystem is active with new tools and fixes, and the k6 Champions Program is recognizing community contributions. Users can access the latest features by downloading the update or joining Grafana Cloud's free plan.

Kubernetes offers significant flexibility and scalability for managing containerized applications, but without careful resource management, these benefits can lead to unexpected costs. To optimize costs effectively, it is crucial to implement fine-grained resource quotas, which help in preventing resource over-allocation by setting precise CPU and memory limits. Additionally, employing both horizontal and vertical pod autoscaling ensures optimal resource allocation and responsiveness to workload changes, while predictive autoscaling can further enhance resource management by using historical data to forecast demand. Dynamic storage optimization and the fine-tuning of resource requests and limits are also essential practices, as they help in maintaining data accessibility and preventing excessive resource consumption. Establishing a culture of proactive cost governance within engineering teams encourages cost-efficient practices, and utilizing tools like Grafana Cloud's Kubernetes Monitoring solution allows for better visibility into resource usage and cost analysis. This comprehensive approach ensures a balanced, efficient, and cost-effective Kubernetes environment.

Grafana is an open-source tool for visualizing and monitoring data from various sources, offering multiple display options like time series graphs, heatmaps, and 3D charts. A common task in Grafana is retrieving the latest value from a data range, which can be achieved using instant queries for efficiency, as they return a single value rather than multiple data points. This is particularly useful when using the stat panel, which highlights the latest or current value with a colorful background and optional sparkline, allowing users to see a small time series graph in the background. Challenges, such as seeing "N/A" in visualizations, can occur if no data points exist within the selected time range, but can be mitigated by adjusting the time range, using the query inspector, or setting custom "No Value" options. The article encourages users to explore Grafana's capabilities, join discussions in the community forum, and consider trying Grafana Cloud for advanced observability features.

Grafana Labs released Grafana Image Renderer v3.8.3 to address a high-severity vulnerability, CVE-2023-4863, which allowed remote attackers to perform an out-of-bounds memory write via a crafted HTML page due to a flaw in the libwebp library used by Google Chrome. This vulnerability impacted all versions of Grafana running the Image Renderer plugin v3.8.2 or earlier, as the service uses Chrome for rendering dashboards. The issue was discovered after Google released a Chrome update addressing the flaw, and the Grafana team quickly implemented a fix in version 3.8.3, which was promptly deployed to Grafana Cloud. Grafana Labs ensured that cloud providers offering Grafana Cloud Pro were notified under embargo and confirmed their services' security. Customers were advised to upgrade to the latest version, and Grafana Labs emphasized responsible disclosure practices, requesting that vulnerabilities not be disclosed publicly until a fix is available and announced.

In a detailed exploration of distributed systems and their complexities, the narrative revolves around the fictitious QuickPizza application, highlighting an incident caused by service dependency issues due to increased load. The engineering team, faced with service timeouts, utilized xk6-disruptor, an extension for Grafana k6, to enhance their testing strategies by simulating faults and reproducing conditions that led to the failure. This tool enabled them to implement a retry and backoff mechanism, effectively resolving the issue and allowing for more robust handling of service disruptions. The team demonstrated the benefits of integrating fault injection into their testing suite, which led to a deeper understanding of failure propagation and improved overall system reliability. The success of these measures was validated through testing, which showed a significant reduction in request failures, underscoring the efficacy of using xk6-disruptor for proactive incident management in distributed systems.

Grafana Labs places significant emphasis on understanding and nurturing its open-source community, recognizing it as a vital component of their success. The company adopts a model to gauge community engagement based on the 90/9/1 rule, which highlights participation inequality, and uses a tool called Common Room to measure community activity through impact points. Although their current active community comprises 71,366 users, Grafana Labs acknowledges the need to increase the number of advanced contributors and improve community engagement. To address this, they are enhancing documentation and tutorials, launching a Champions Program to recognize top contributors, and streamlining their online presence to better consolidate community interactions. The company values transparency and feedback, encouraging community input to further improve their open-source ecosystem. Grafana Labs' commitment to open-source principles is reflected in their hiring practices and company culture, which prioritize shared growth and open communication.

Grafana Labs has released new versions, 10.1.5, 10.0.9, 9.5.13, and 9.4.17, to address a medium-severity security vulnerability, CVE-2023-4822, in the role-based access control (RBAC) system of Grafana Enterprise. This vulnerability allows organization administrators to alter permissions across different organizations, posing a risk in instances running multiple organizations, though Grafana Cloud instances are unaffected. The issue, with a CVSS score of 6.7, was identified in versions 8.0.0 to 10.1.4, and users are advised to upgrade to the patched versions or limit administrator privileges to trusted individuals if immediate upgrading is not feasible. The timeline reveals the vulnerability's discovery and patching process, culminating in a public release on October 12, 2023. Grafana Labs encourages reporting of security vulnerabilities via their dedicated page, urging discretion until a public announcement is made.

Grot is an AI-powered chatbot created by Grafana Labs in collaboration with Pal to assist users in navigating and utilizing Grafana's platform effectively. It is designed to answer a wide range of questions, from basic inquiries about Grafana's LGTM Stack to more technical issues like configuring data sources and troubleshooting. Grot leverages large language models trained on a variety of Grafana Labs resources, including documentation and blogs, to provide accurate and conversational responses in multiple languages. Currently in beta, Grot aims to offer a user-friendly experience with the ability to clarify and iterate on answers based on user feedback. Users can interact with Grot by clicking the search icon on Grafana's website, and they are encouraged to provide feedback to help improve the chatbot's accuracy and functionality.

AWS's serverless technologies, such as Lambda, offer cost-effective scaling and separation of concerns but pose challenges in monitoring due to their transient and abstracted nature. To address this, AWS developed integrated solutions like Amazon CloudWatch and AWS X-Ray, and later introduced the AWS Distribution for OpenTelemetry (ADOT) to facilitate vendor-neutral instrumentation and data export. This text describes how users can instrument Lambda functions with OpenTelemetry and send traces to Grafana Cloud using Grafana Agent as an OTLP endpoint, ensuring scalability and reliability. The process involves creating a Lambda function, adding instrumentation through a Lambda layer, and configuring the Grafana Agent to relay data. The use of OpenTelemetry provides insights into serverless performance, enabling optimization and troubleshooting. Users are encouraged to provide feedback and explore Grafana Cloud’s free tier, which includes various metrics and tracing capabilities.

Grafana dashboards are versatile tools for monitoring applications and infrastructure, and there are several methods to embed them into web applications, including links, snapshots, and public dashboards, each with its own set of advantages and limitations. Links are simple and maintain data security but aren't true embedding, while snapshots provide interactivity without impacting data sources but require public accessibility, and public dashboards offer full interactivity but expose data to anyone with the link. The article emphasizes the importance of understanding security implications, such as clickjacking and cross-site request forgery, when embedding dashboards, especially in terms of configuring iframe tags and managing cookies. Embedding fully authenticated dashboards is not supported in Grafana Cloud due to user experience and security challenges, although alternative solutions like using OAuth or proxying through a trusted service are possible but complex. For those requiring more advanced features, self-managed environments like Grafana OSS or Grafana Enterprise offer greater flexibility compared to the managed Grafana Cloud service.

Grafana Agent v0.37 introduces significant updates, achieving feature parity between its Static and Flow modes, which simplifies the transition to Flow mode through an enhanced config converter. This release allows for the splitting of Flow configurations into multiple files, enhancing ease of use by maintaining manageable config sizes and reducing Git conflicts. Grafana Agent Flow, which includes a custom configuration language called River, is set to replace Static mode as the default, offering improved usability, debuggability, and adaptability for power users. The update also includes a variety of new components for discovery and processing, such as those for Kubernetes, Docker Swarm, and various cloud APIs, facilitating comprehensive telemetry data handling. While the conversion from Static to Flow mode is supported, users are encouraged to review converted configurations before deploying them in production environments, as some configurations might not yet be fully supported for conversion.

Grafana Labs is celebrating the 10th anniversary of Hacktoberfest, inviting participants to join the global open-source community in contributing to essential projects like Grafana, Loki, and Prometheus. Hacktoberfest occurs every October and is an opportunity for individuals of all experience levels to enhance their skills, collaborate with experienced engineers, and possibly find mentors, while also contributing to widely used tools such as Linux Kernel, Kubernetes, and VS Code. Founded by DigitalOcean, this year's Hacktoberfest includes global coding events, and Grafana Labs is actively participating by highlighting specific issues in their repositories for contributors to tackle. Participants can register on the Hacktoberfest website and are encouraged to submit pull requests to have a tree planted in their name or receive a digital reward. Grafana Labs underscores the importance of open-source software in democratizing metrics and making observability tools accessible, with a special focus on collaboration and community contributions throughout October.

Grafana Cloud now supports AWS PrivateLink, allowing for secure transmission of telemetry data from AWS-hosted services directly to Grafana Cloud instances without exposing traffic to the public internet. This integration, available in all regions hosting Grafana Cloud, enables organizations to send metrics, logs, traces, and profiles securely, eliminating egress traffic fees and reducing overall costs. AWS PrivateLink uses VPC endpoints on both consumer and provider sides to facilitate private connections within the AWS infrastructure, requiring additional steps like VPC peering for inter-region setups. The service works alongside Private Data Source Connect (PDC), which queries network-secured data sources, to provide a comprehensive observability stack in Grafana Cloud. AWS PrivateLink is now accessible to all Grafana Cloud users, including those on the free tier, and additional resources and guides are available for configuration, with future insights to be shared at ObservabilityCON 2023.

Grafana Cloud has introduced Private Data Source Connect (PDC), now generally available to all users, allowing secure querying of network-secured data sources like MySQL databases or Elasticsearch clusters from within a private network or Virtual Private Cloud without exposing them to the internet. This feature establishes a secure link via an SSH tunnel, enabling users to visualize and alert on private data without needing to allowlist numerous IP addresses, thus enhancing security and operational efficiency. PDC supports multiple data sources within the same network and allows for connections between a single Grafana Cloud stack and multiple private networks, facilitating comprehensive data integration. The service has been welcomed by organizations like Infodation and Notino for its robust security and ease of setup, and it complements other Grafana Cloud tools like continuous profiling and performance testing. While PDC focuses on querying data sources from Grafana Cloud, it is distinct from AWS PrivateLink, which provides private connectivity for telemetry data between AWS VPCs and Grafana Cloud. The release underscores Grafana Cloud's commitment to providing secure, efficient, and comprehensive observability solutions, with future enhancements planned to further expand its capabilities.

ObservabilityCON 2023, scheduled for November 14-15 in London, is an open-source observability event that features community-led sessions highlighting unique observability journeys and success stories from industry experts. The agenda includes discussions on full stack observability, real user monitoring, Kubernetes monitoring, and strategies for supporting hybrid and multi-cloud environments with the Grafana LGTM Stack. Notable sessions include ComplyAdvantage’s observability platform overhaul, Maersk's consolidation of tools using the Grafana LGTM Stack, and Pipedrive's migration to OpenTelemetry and Grafana offerings. The event will also feature technical workshops, a keynote by Grafana Labs leaders, and opportunities to connect with peers. Participants are encouraged to register early for discounted pricing and to take part in Grafana Labs' annual observability survey.

Continuous profiling using Grafana Cloud Profiles significantly enhanced code performance for a new feature in Grafana Loki, an open-source log aggregation system. The author, initially unfamiliar with continuous profiling tools, used Grafana Cloud Profiles, powered by Pyroscope, to analyze CPU usage and discovered that excessive time was spent on fmt.Sprintf calls within a loop. By moving these calls outside the loop, the application's runtime was reduced by 30%, demonstrating the tool's effectiveness in optimizing performance. Despite the author's lack of prior experience, Grafana Cloud Profiles proved intuitive and efficient, allowing for substantial improvements in under 10 minutes. This experience highlighted the potential of continuous profiling to streamline development processes and optimize future features, encouraging the integration of such tools into routine workflows.

Sentry Software, a small company focused on sustainability, has successfully reduced its carbon emissions by 7% year over year through innovative use of open-source technologies and open standards. By developing a monitoring system using Grafana dashboards and OpenTelemetry guidelines, they efficiently tracked and reduced electricity usage at their primary data center in Paris, achieving a 19% reduction in AC energy consumption. The company re-engineered its proprietary Hardware Sentry tool as a Prometheus exporter and utilized custom scripts to gather detailed data on energy consumption and carbon emissions. This allowed them to make informed decisions, such as gradually increasing the data center's temperature to 27 degrees Celsius, resulting in significant energy savings. Sentry Software's approach demonstrates how open-source solutions can empower organizations to effectively monitor and reduce their IT infrastructure's carbon footprint, offering their tools for free to encourage others to follow suit in the journey toward carbon neutrality.