April 2020 Summaries

Grafana 101 is a webinar series that covers various aspects of Grafana, including creating dashboards and visualizations. The first session, "Creating Awesome Visualizations," demonstrates how to build six common visuals for different scenarios, such as DevOps, IoT, geospatial data, and public datasets. The session also provides tips and tricks for using variables, series-override, and thresholds to create interactive and useful dashboards. The recording and slides are available online, and the next session in the series will focus on getting started with alerting. The webinar aims to help users learn how to use Grafana to build dashboards and visualize data, making it easier for developers to understand when issues arise in their critical metrics and take action quickly.

TimescaleDB has released version 1.7 with several updates including PostgreSQL 12 support and Real-time Aggregates feature that returns real-time results for continuous aggregates over selected time intervals, providing faster query performance by combining pre-computed hourly rollups with raw data from the last hour. The new release also includes data management features available in the free Community edition, such as data re-ordering and retention policies, to simplify data management. The update aims to improve the overall performance and usability of TimescaleDB for users.

Compression algorithms, such as delta-delta encoding, Simple-8b, and XOR-based methods, play a crucial role in efficiently managing time-series data by significantly reducing storage costs and enhancing query speeds. These algorithms allow data to be encoded using fewer bits than its original representation, with techniques like delta-encoding and delta-of-delta encoding focusing on storing only the differences between data points, particularly effective for time-series data that changes gradually. Simple-8b further optimizes storage by encoding variable-length integers in fixed-size blocks, while run-length encoding compresses repeated values efficiently. XOR-based compression, as used in the Gorilla database, provides effective compression for floating-point numbers by storing only differing bits between successive values. TimescaleDB, an open-source time-series database, implements these algorithms to achieve over 90% storage efficiency, employing different compression techniques based on data type and enabling faster queries through reverse data decompression. The strategic use of these algorithms not only leads to substantial cost savings but also enhances compute performance by reducing the data footprint in storage systems.

The SolarNetwork Foundation, a New Zealand-registered charity organisation, aims to promote renewable energy and sustainability worldwide by providing an easy-to-use technology platform that tracks energy usage and provides measurable facts. The foundation's platform, SolarNetwork, collects various data types such as electrical energy measurements, weather data, and market price data, allowing users to view their energy consumption in clear and simple ways. The team behind SolarNetwork uses TimescaleDB for database management, which has provided flexibility and efficiency in managing large time-series datasets. The foundation's work has helped many organisations, including schools and sustainability-focused businesses, track their energy use and production, reducing non-renewable energy consumption and promoting renewable generation.

The Timescale Community is launching a "Write for Timescale" program, which invites software engineers, data scientists, and related professionals to submit technical tutorials on using TimescaleDB. The program pays market rate for tutorials that showcase the features of TimescaleDB in an interesting and educational way. Submissions should be well-written, provide detailed technical walkthroughs, and demonstrate original work. The tutorials can use public datasets or existing datasets to solve problems or gain insight into a topic of general interest. The submissions will undergo editing and approval before payment is issued. The program aims to share the power of data with the world through these tutorials.

The text provides a detailed guide on creating interactive dashboards using Grafana, an open-source visualization tool, to enhance usability for non-technical stakeholders. It highlights the common issue of static dashboards that require engineers to make code changes for detailed insights and proposes a solution through Grafana's variables feature. By using Grafana's UI, users can incorporate interactive elements like "pickers" or filters, enabling stakeholders to manipulate data without altering the underlying SQL queries. The example of monitoring live bus locations in New York City is used to illustrate the process of creating a variable that controls the visualization of different bus routes. The guide explains how to define a SQL query that generates a variable and modify the graph query to incorporate this variable, allowing users to select from multiple options or all routes, resulting in a dynamic visual representation that updates based on user selections. The guide concludes with additional resources for building Grafana dashboards, offering comprehensive tutorials and tips for visualizing data effectively.

The problem of distorted trends on the same graph is a common issue when plotting multiple variables, particularly when their scales differ significantly. To solve this problem, Grafana's series-override feature can be used to create two separate Y axes for each variable, allowing for more accurate representation of both trends without distorting one another. This solution enables users to compare and contrast different data points in a single graph, providing valuable insights into complex relationships between variables.

Our guest speakers covered various topics related to wrangling public data, including techniques for handling geospatial images, writing faster queries, and using open data to monitor COVID-19 and climate change. The first session of DataPub, a new monthly meetup focused on open data, was hosted online, allowing more people to join and share diverse perspectives. Guest speakers Joel Natividad, Saheel Ahmed, and Jonan Scheffler shared their experiences with wrangling public datasets, including using satellite imagery for environmental monitoring and optimizing geoTIFFs for faster queries. The event aimed to help community members find great public datasets, better interpret results, and use open data to drive positive change.

We provide a biweekly newsletter sharing awesome technical content and resources for developers to do amazing things with their data, including new documentation, tutorials, code samples, virtual sessions, and community spotlights. We're on a mission to teach the world about time-series data and support growing communities worldwide by sharing educational resources broadly. Our latest content includes guides on using open datasets and tools like Prometheus and Grafana, tutorials on monitoring personal projects, and analysis of COVID-19 data using TimescaleDB. We also host virtual events, community meetups, and office hours to help developers get the most out of our products and services.