June 2019 Summaries

Alexander Zobnin, a fullstack developer at Grafana Labs, is based in St. Petersburg, Russia, and primarily works on frontend tasks and plugin development, notably creating and maintaining the Zabbix plugin for Grafana. Outside of his professional life, Zobnin enjoys sports such as mountain biking and running, spending time with his family, and pursuing photography, video editing, and playing the guitar. He prefers coding in silence and uses two spaces for indentation. His favorite tech gadget is his GoPro camera, which he uses to capture memorable moments. His GitHub handle is alexanderzobnin, and he can be found on Twitter under the same username.

Grafana is evolving from a general dashboarding tool into a comprehensive observability platform, with new features tailored for DevOps, as highlighted by Grafana Labs’ Director of UX, David Kaltschmidt, at the 2019 InfluxDays Conference. A notable feature is the Explore mode, introduced in Grafana version 6.0, which allows users to modify queries without altering dashboards, enhancing the troubleshooting process for engineers. This mode supports visualizing both metrics and logs from Influx, a time series database that uniquely stores both numeric and string values. The integration of log aggregation is essential as it allows engineers to investigate alerts through centralized log storage, preventing data loss if machines fail. InfluxDB's capability to tag and label logs facilitates the isolation and filtering of relevant data, optimizing the troubleshooting process. Logs can be ingested into Influx via its API, Telegraf plugins, or Fluentd, while Grafana's Explore mode further enhances log data exploration with features like tag-based filtering and split view for side-by-side metrics and logs analysis. Future developments aim to unify the Influx and Flux datasources, simplifying the user experience, as Grafana continues to focus on log aggregation as a core aspect of its observability strategy.

The Grafana community spotlight highlights the BigQuery datasource plugin developed by DoiT International, a company that assists businesses in transitioning to or between cloud services like Google Cloud and AWS. Aviv Laufer, a Principal Reliability Engineer at DoiT, created the plugin to address the demand for a Grafana connector to BigQuery, as many customers were already using Grafana for data visualization instead of relying on Google’s native tools. Despite being his first time coding in TypeScript and developing a Grafana plugin, Laufer managed to release it successfully after overcoming challenges related to limited documentation. Since its release, the plugin has been adopted by several of DoiT's customers and other users globally, who have contributed feedback and requested additional features such as alerts. Laufer and the DoiT team are committed to enhancing the plugin based on community feedback and are open to developing more plugins that could benefit their customers and the wider community, emphasizing their dedication to open-source solutions.

GitLab is renowned for its radical transparency, exemplified by its public sharing of internal performance metrics through Grafana dashboards, a practice highlighted by Staff Backend Engineer Ben Kochie at GrafanaCon. Initially, GitLab used a simple Ruby script to synchronize internal and public dashboards, but as their system grew, they faced challenges with Prometheus scaling and query management, leading them to implement horizontal sharding and the Thanos query layer for better efficiency. To enhance caching and performance, GitLab integrated Trickster, a reverse proxy server, while also addressing potential data privacy risks through careful management of feature flags to prevent data leaks. Despite occasional system strain from heavy dashboard usage, GitLab continues to refine its infrastructure, aiming to incorporate the Thanos storage engine for improved query performance, and embraces the public engagement their dashboards receive.

Marcus Efraimsson is a versatile developer at Grafana Labs, primarily focusing on backend work while occasionally engaging in frontend activities. Based in Stockholm, he is involved in making Grafana's Explore logs visualization more adaptable to various data sources beyond Loki, such as InfluxDB and Elasticsearch. Marcus is passionate about enhancing contribution processes for Grafana projects and can be found on GitHub as marefr and on Twitter as @mefraimsson. In his leisure time, he enjoys socializing, traveling, watching sports and TV series, and has recently appreciated a foldable height-adjustable table for his home office. Marcus's recent binge-watch was the Danish series "Follow the Money," and he humorously identifies with the character Jack Bauer from "Game of Thrones."

Synthetic monitoring offers a user-centric perspective on service health by simulating user interactions and capturing metrics that reflect the end-user experience, rather than solely relying on traditional metrics like CPU and memory usage, which may not accurately represent system functionality. The tutorial by Brian Gann demonstrates using hosted Grafana to set up synthetic monitoring through a Python script that simulates a 10-step login and validation process, capturing metrics like connection latency, response duration, and success state. These metrics are formatted for Graphite but can be adapted for other databases like InfluxDB, providing insights into potential bottlenecks and helping ensure a service remains operational from a user's perspective, even if some components are degraded. The process involves using tools like Chrome Developer tools and Postman to identify each step, and the resulting data is visualized in Grafana dashboards, enabling service reliability engineers to optimize and maintain the health of applications effectively.

Grafana Labs is focused on democratizing metrics by providing a unified platform that empowers users across various sectors, from large corporations to individual enthusiasts, to easily access, visualize, and manipulate data without needing extensive technical expertise. This open-source tool leverages time projects like Prometheus and InfluxDB to support a self-service model, enhancing information sharing and decision-making within organizations. Grafana's point-and-click interface allows users to manage dashboards and interact with data dynamically, reducing reliance on specialized personnel. Additionally, Grafana fosters community collaboration with over 600 developers contributing to its continuous improvement and adaptability to various data sources, including Elasticsearch and Web APIs. This collaborative approach ensures that Grafana remains at the forefront of observability solutions, enabling quicker problem-solving and broader data accessibility.

Peter Holmberg, a frontend developer at Grafana Labs based in Stockholm, Sweden, primarily works on the grafana.com website to improve user experience in finding information and dashboards. Although he has contributed to open-source projects in the past, his current focus is on enhancing the website's functionality. In his free time, Peter enjoys cycling, playing ice hockey, and has recently built a gaming PC to rekindle his interest in gaming, currently playing Fallout76. He also indulges in watching a Swedish '90s soap opera, "Rederiet," despite considering it a guilty pleasure. His GitHub and Twitter handles are peterholmberg and @peteholmberg, respectively.

AMMP Technologies, which specializes in monitoring energy systems, particularly mini-grids in Africa, utilizes Grafana to effectively monitor real-life streaming data, demonstrating how such tools designed for computer metrics can be adapted for industrial IoT applications. During GrafanaCon L.A., co-founder Svet Bajlekov highlighted the importance of transitioning to open-source technology stacks for industrial IoT to enhance security and flexibility, especially as traditional systems like SCADA are outdated and increasingly vulnerable to cyber threats. Bajlekov advocates for embracing internet protocols and open interfaces to build secure, extensible, and interoperable systems, leveraging tools like Grafana for visualization, LoudML for analytics, and EdgeX Foundry for IoT edge computing. While the transition to such open-source models faces resistance, particularly from conservative utility companies, Bajlekov remains optimistic about the potential for a more open and interconnected technological ecosystem that can effectively manage and secure industrial operations.

Blerim Sheqa, the CPO of Icinga, delivered a talk at GrafanaCon LA about avoiding common pitfalls in data visualization, stressing the importance of conventions, proper labeling, comparability, readability, and understanding of data. He illustrated how incorrect data visualization can lead to misinterpretations by using examples such as a misleading graph about gun laws and a poorly labeled load graph. Sheqa emphasized that following conventions, using proper labels, and ensuring graphs are readable and comparable are crucial for effective visualization, particularly in debugging infrastructure issues. Highlighting key parts of a graph, using grids, and avoiding overcrowding with too many annotations can improve clarity and understanding. He also underscored the necessity of knowing the data thoroughly to prevent misinterpretation and create accurate and useful dashboards.

Dominik Prokop is a frontend engineer at Grafana Labs, based in Warsaw, Poland, focusing on React migration and the @grafana/ui, and also involved with Explore. Though new to the open-source community, Dominik contributes by reporting issues and bug fixes, and his GitHub handle is dprokop. In his free time, he enjoys photography, architecture, interior design, and baking, and has a minimalist approach to technology, recently appreciating the Sonoff basic smart switch for home automation. He enjoys watching the "Street Food" documentary series and prefers to code in silence, occasionally listening to music like techno or Nils Frahm when working remotely.

At the Monitorama Conference, Dave Cadwallader shared his insights on observability through a unique approach of teaching his children about monitoring using Grafana. By engaging with his kids through practical experiments with inexpensive temperature and humidity sensors, and later an ultrasonic range finder, Cadwallader illustrated how monitoring could be made accessible and engaging for all ages. These experiments not only introduced his children to concepts like data visualization and historical data analysis but also provided Cadwallader with a fresh perspective on teaching and collaboration in professional environments. He emphasized the importance of intrinsic motivation and experiential learning, suggesting that allowing individuals to explore and experiment on their own fosters a deeper understanding and enthusiasm for tools like Grafana. Cadwallader's experience culminated in organizing a game day at his workplace to encourage playful learning, reinforcing the idea that enjoyment enhances the learning process.

TimescaleDB, an open-source database extension for PostgreSQL, enhances time series data management while maintaining compatibility with the PostgreSQL ecosystem, allowing users to utilize functions like JSON indexes and relational tables. The integration of TimescaleDB with Grafana through the PostgreSQL query builder simplifies time series data exploration by offering a user-friendly visual query editor, which improves accessibility for users with varying SQL proficiency levels. Despite SQL's traditional limitations in scaling for time series data, TimescaleDB addresses these challenges by implementing a unique abstraction layer that partitions data into chunks based on time intervals, optimizing performance and enabling efficient data handling within memory. This approach allows TimescaleDB to support both relational and time series databases seamlessly, offering improved performance without significant degradation in data insertion rates. The enduring popularity of SQL, as evidenced by its widespread use among developers and business analysts, underscores the practical benefits of integrating SQL into time series monitoring systems, despite the traditional scaling concerns associated with SQL databases.