May 2022 Summaries

Telegraf, a key data collection agent from InfluxData, has been made easier to use with the launch of the Data Collection with Telegraf course as part of InfluxDB University. This self-paced online course covers essential topics such as data collection, best practices, and custom plugin creation, taught by product leaders and software engineers who focus on Telegraf. The course is designed for users looking to become Telegraf experts and offers expert troubleshooting advice to avoid common mistakes. Upon completion, learners will earn a digital badge to share on their social or professional profiles.

InfluxDB Enterprise 1.9.7 has been released with several enhancements, including the exposure of passive nodes to influxd-ctl command for increased horizontal scaling, throttling of replication and hinted hand-off when errors occur, and improvements to Flux, a query language used in InfluxDB. The release also includes performance enhancements, such as continued optimization of sort() and map(), and bug fixes. Additionally, the upgrade to Go 1.17.9 and Flux v0.161.0 has been implemented. This release is primarily aimed at InfluxDB Enterprise customers, with no corresponding OSS 1.9 release, but users can still build InfluxDB 1.x from source code if needed.

In this latest update, InfluxData has introduced several features aimed at improving the developer experience and productivity. The new Help & Support tab provides an ever-present help experience for cloud users directly in the UI, allowing them to find relevant content faster. The Onboarding Wizard helps application builders get setup within their first 5 minutes of interacting with InfluxDB Cloud, streamlining the onboarding process. Additionally, deep linking enables easier workflows by allowing users to jump directly to specific pages. Other notable features include UI color mapping, Flux dynamic help panels, post-query statistics, and bug fixes that address various issues such as date functions in Flux working correctly with timezones enabled and security updates to render _blank links from ReactMarkdown.

In a project led by Herrenknecht AG, an industrial technology leader in mechanized tunneling systems, a team of engineers aimed to build an Industrial Internet of Things (IIoT) platform to provide real-time data insights for their massive tunnel boring machines. These machines have thousands of sensors generating high-velocity data and require current and accurate information to operate safely and efficiently. Herrenknecht selected InfluxDB as the central time series storage database due to its ability to handle edge device data collection, offer open source and enterprise versions, and support Windows. The team initially installed the open-source version on the machines and later built a cloud ecosystem using InfluxDB OSS, which reduced costs by sending clean data to the cloud instead of storing it locally. After reaching 100 instances, they migrated from open-source to InfluxDB Enterprise to improve reliability and stability, resulting in cost savings and better performance for their IIoT platform.

InfluxDB is used to store and analyze Bitcoin price data from the CoinMarketCap API, allowing users to calculate a simple Momentum signal. The signal is generated by crossing the 1-day moving average over the 5-day moving average, resulting in a buy or sell signal when the 1-day MA crosses above or below the 5-day MA. A Telegraf agent is used to collect data from the API and store it in InfluxDB, while Flux is used to calculate the Momentum signal and set up alerts for specific conditions. The system includes an alerting feature that sends notifications to a Slack channel when the momentum signal reaches a certain threshold, allowing users to stay informed about market trends and make trading decisions based on automated analysis.

The Particle platform is a comprehensive IoT solution that enables companies to manage and program their devices, bringing them to market faster. To address the challenge of collecting telemetry data from large numbers of edge IoT devices, Particle developers built a solution using Kubernetes, Telegraf, InfluxDB, and Flux. The resulting Fleet Health console provides users with granular device-specific data, allowing them to identify issues quicker and improve their development cycle. With its user-friendly UI and ability to scale from prototype to enterprise, the Particle platform helps companies gain greater visibility into the performance of their edge IoT devices.

Prescient Devices is a company that focuses on changing the way businesses think about and use edge data and industrial IoT. It has built an edge data solutions platform called Prescient Designer, which empowers organizations to easily design and orchestrate edge-to-cloud data solutions using Node-RED's low-code visual interface and InfluxDB Cloud for storage. The company chose InfluxDB as its backbone due to its open-source nature, scalability, and ability to handle the scale and speed of data generated from large fleets of edge IoT devices. Prescient Designer supports custom brokers for users with specific compliance requirements, such as HIPAA and GDPR, and enables non-developers to build IIoT management applications through its intuitive visual builder, accelerating implementation by twelve-fold and lowering costs six-fold.

The InfluxDB team has introduced new optimizations to the aggregateWindow() function to improve its performance, particularly for users querying large amounts of data or having high cardinality data. These changes enable grouping like series together before calculating aggregates and allow pushdown patterns to be used more efficiently, resulting in improved query performance. The updated function is now implemented in native Go for specific functions, such as mean, sum, count, and min aggregation, making it even more performant for these use cases. To maximize the benefits of the aggregateWindow() function, users should ensure that the createEmpty option is set to false. The performance improvements have been demonstrated through testing against 50,000 points with a cardinality of 50, resulting in a significant reduction in execution time from ~508 ms to ~200 ms for a specific query pattern.

Telegraf has released version 1.22.4, which includes updates to the agent service and input/output plugins. The Telegraf service now waits for network up in systemd packaging, and the Couchbase Input plugin no longer assumes bucket stat metrics will all be the same length. Additionally, the Statsd Input plugin is fixed to no longer error when closing a statsd network connection. Updates have also been made to the Azure Monitor Output and Wavefront Output plugins, including checking for context deadline errors and fixing empty tag issues. The release includes updated dependencies, such as github.com/showwin/speedtest-go and OpenTelemetry plugins. Users can download the latest Telegraf release from the InfluxDB website and report any issues or questions to the InfluxDB Community Slack or GitHub Repo.

Scott Anderson, a father, shares an experience of his 9-year-old daughter conducting a science experiment on whether playing video games affects blood pressure. She collected data by measuring and recording the blood pressure of willing participants before and after playing Mario Kart 8 for 7 minutes. With the help of her father, who is familiar with InfluxDB, they converted the handwritten results into line protocol and built queries to answer questions such as what was the average change in blood pressure overall and if sex or age mattered. The final dashboard provided a visual representation of the data, including percentage changes, which helped her draw conclusions about the experiment's findings, including a 2% drop in blood pressure on average, with males experiencing a higher drop than females, and younger subjects showing less change in blood pressure compared to older subjects.

In this article, Anais Dotis-Georgiou explains how to use Flux to bring in data from various sources, including SQL databases and JSON objects. She provides four examples of using Flux with increasingly complex JSON types, including extracting top-level values, mapping across arrays, handling nested arrays, and evaluating whether values exist or not. The article also covers an example using the OpenWeatherMap API to gather weather data and write it to InfluxDB. With these examples, users can learn how to use Flux to manipulate JSON objects and create tables for ingestion into InfluxDB.

InfluxData will sponsor and exhibit at KubeCon + CloudNativeCon Europe 2022, where experts from InfluxData will conduct live demonstrations of InfluxDB and share technical expertise on scaling Kubernetes deployments with effective disaster recovery processes. The company is a Silver sponsor of the event and has one of the world's largest Kubernetes users. InfluxDB Cloud is a large commercially available service using Kubernetes, providing a cloud abstraction layer for developers to deploy new cloud regions faster and onboard new customers quickly. InfluxData experts will host various sessions throughout the conference, including a session on disaster recovery best practices for users with customer-facing workloads on Kubernetes.

InfluxData is a company that offers time series data management tools, and Susannah Brodnitz has joined their marketing team after working as a research assistant in oceanographic studies. She was drawn to the company because of its focus on using clear and engaging writing to help people understand complex ideas and information. The role involves learning about various uses of InfluxDB and writing about them in an exciting way, which sparks new projects. Susannah is interested in the company's product, particularly its geospatial analysis tools that can significantly reduce analysis time. She values the company's commitment to sharing code and working in a collaborative team environment, which she finds engaging and welcoming after her isolated academic research experience.

InfluxData is preparing to attend KubeCon EU 2022, where they will be available at booth G6 to discuss Kubernetes, InfluxDB, and application development. Before the event, attendees can access various resources such as technical papers and blog posts to get started with InfluxDB and Kubernetes. InfluxData will also present a session on effective disaster recovery and have sales engineers on-site to provide guidance and resources for implementing Kubernetes and InfluxDB in new projects. Additionally, live demos showcasing the integration of InfluxDB and Kubernetes will be available at their booth, along with swag such as socks and stickers.

InfluxDB is a time-series database used for storing IoT data, while MQTT is a lightweight messaging protocol used for efficient communication between devices in IoT environments. A smart sensor can be created using Python to transmit measurements over the internet using MQTT, which stores the data in InfluxDB. The MQTT broker acts as an intermediary for pub/sub (publish and subscribe) communication between devices and applications. InfluxDB provides a database and UI tools that enable visualization and querying of IoT data. By creating a smart sensor and setting up InfluxDB, users can efficiently store and manage IoT metrics, such as temperature measurements, in real-time. The system also includes an MQTT consumer that receives messages from the broker, stores them in InfluxDB, and visualizes the data using the InfluxDB UI.

Telegraf has released version 1.22.3, addressing security concerns with an updated Go version and fixing issues in the GNMI input plugin. The InfluxDB listener input plugin has been improved by removing duplicate writes, while the system input has reduced log levels for disk plugins. Users can download the latest release from the Telegraf website and report any issues or questions to the InfluxDB Community Slack or GitHub repo. Interested users can also participate in testing bug fixes and features by filling out a form on the Telegraf website.