September 2024 Summaries

Siemens Energy has standardized its predictive maintenance operations on InfluxDB, a time series database by InfluxData. The global energy leader uses InfluxDB to optimize data collection and analysis across its energy storage operations, allowing it to gather high-frequency, high-resolution sensor data in real-time for advanced monitoring and control systems. Siemens Energy's adoption of commercial InfluxDB has enabled the company to scale its data operations significantly, managing 700 high-volume write requests and 800 real-time queries per minute across research and development labs and production cells.

The article discusses the use of Node-RED, time series databases (TSDBs), and Grafana in managing and visualizing data from Industrial Internet of Things (IIoT) devices. Node-RED is an open source tool for connecting IIoT devices with APIs and online services using minimal coding. TSDBs are specialized systems that efficiently handle time-stamped or time series data, which is useful in IoT environments where changes over time are essential. Grafana is an open source monitoring and observability platform that excels at visualizing time series data, allowing users to create dynamic and interactive dashboards. The integration of these tools provides a complete solution for managing and visualizing IIoT data, enabling real-time monitoring, predictive maintenance strategies, and efficient handling of large data streams.

InfluxDB has adjusted how it handles "partial writes" in its InfluxDB Cloud Serverless product using the v2 Write API, affecting customers who created their accounts after January 31, 2023. Partial writes will now always return a 400 status code. The most common condition causing partial writes is an error in the Line Protocol's syntax. InfluxDB plans to make this change for InfluxDB Cloud Dedicated and Clustered customers as well. A v3 Write API is also planned, which will revisit handling of partial writes and add new options and data types.

This article discusses deploying InfluxDB OSS 2.x and Telegraf to monitor Kubernetes clusters. The author explains the benefits of using InfluxDB and Telegraf, including easy off-site replication, ease of setup, and familiarity with the tools. They also provide a step-by-step guide on how to create a dedicated namespace for monitoring, deploy and preconfigure an InfluxDB OSS 2.x instance, and configure Telegraf as a DaemonSet. The article concludes by discussing how to visualize metrics in Grafana and provides tips on dashboarding.

The latest release of Telegraf, version 1.32, introduces new plugins and important changes to its logging framework. New plugins include SLURM workload manager input (inputs.slurm), Parquet file writer output (outputs.parquet), and Remote file output (outputs.remotefile). Changes to the logging framework include a log-level selection for external programs, a disk-backed metric buffer for outputs, and an experimental feature for agent enhancements. The release also includes improvements in configuration verification and introduces InfluxDB University's Data Collection with Telegraf course.

The Industrial Internet of Things (IIoT) presents numerous benefits such as minimized downtime, data-driven decision making, safety and compliance, cost reduction and resource optimization, enhanced customer satisfaction, and innovation and new business models. However, it also comes with challenges including security vulnerabilities and threats, data privacy and compliance, scalability and network latency, skills gap, and legacy systems. Common attack vectors and types of threats in IIoT cloud security include device compromise and malware, man-in-the-middle (MITM), denial of service (DoS), phishing and social engineering, insider threats, and supply chain attacks. To address these issues, best practices for IIoT cloud security involve authentication and access control, data encryption, regular updates and patching, network segmentation, continuous monitoring and incident response plan, and security awareness training. Useful tools for IIoT cloud security include endpoint security tools, network security tools, identity and access management (IAM) solutions, continuous monitoring and SIEM solutions, vulnerability management tools, cloud-specific security solutions, and secure data storage solutions like InfluxDB.

This article discusses the integration of Android apps with InfluxDB Cloud Serverless for efficient management and analysis of real-time data generated by apps, games, and other services. The author presents a 'Sensor Logging' use case where an Android Java app captures motion data from accelerometers and gyroscopes in a smartphone and stores it in InfluxDB Cloud. The data is ingested at sub-second speeds and then queried within the app to display a line graph using a third-party charting library. The integration process includes setting up Android Studio, creating an account with InfluxDB, adding necessary Gradle dependencies, initializing the InfluxDB client, writing sensor data to the bucket, querying data from InfluxDB, and finally visualizing the data in the app using a line chart created by the MPAndroidChart library. The author concludes that this setup allows for seamless data logging, querying, and visualization directly within Android apps.

InfluxData has introduced new capabilities in its InfluxDB 3.0 product suite to simplify time series data management at scale. The rebuilt InfluxDB 3.0 core delivers high performance, including unlimited cardinality, high-speed ingest, real-time querying, and superior data compression through native object storage. These enhancements enable developers to analyze more data at higher speeds without compromising efficiency. Additionally, the company has announced the general availability of InfluxDB Clustered for on-premises or private cloud deployments.

InfluxDB 3.0 has seen numerous updates since its initial release, reinforcing its position as the leading time series database. The latest improvements and new features include operational dashboards for cluster performance monitoring, single sign-on support, a management API, and parameterized queries. Additionally, InfluxDB Clustered is now generally available, allowing users to manage large-scale time series workloads in private cloud or on-prem environments. Performance enhancements have been made across multiple vectors, including custom partitioning, improved aggregation and grouping performance, and faster string-intensive query performance. These updates enable companies to store more data while maintaining query performance and reducing storage costs.

InfluxData has announced new capabilities in InfluxDB 3, including faster query performance and management tooling, to simplify time series data management at scale. The company also announced the general availability of InfluxDB Clustered, its self-managed time series database for on-premises or private cloud deployments. These updates aim to power massive time series workloads with high-performance features such as unlimited cardinality, high-speed ingest, real-time querying, and superior data compression through native object storage. The new capabilities are designed to help developers build real-time monitoring and control systems, analyze more data at higher speeds without compromising efficiency, and manage large-scale time series workloads more easily.

In this blog post, the authors discuss how to accelerate string operations using StringViewArray in Arrow and DataFusion. They cover various aspects such as faster comparison, take and filter operations, garbage collection, function inlining, buffer size tuning, end-to-end query performance improvements, and deduplicating string values for aggregation. The authors also mention potential pitfalls of using StringViewArray and express their gratitude to the community members who contributed to this project. Overall, the implementation of StringViewArray has shown significant performance improvements in string-intensive workloads.