February 2025 Summaries

InfluxDB 3 Core and Enterprise offer a streamlined single-node and single-process architecture that enables fast deployment and operation. Built on the FDAP stack, these products provide optimized data storage, processing, and interoperability with external tools and libraries. They feature flexible storage options, including memory, disk, and object storage, and operate in a diskless architecture for simplicity and efficiency. The Processing Engine brings extensibility and control to both Core and Enterprise, empowering users to transform and normalize data, combine multiple sources, trigger alerts and notifications, downsample, replicate data, and more. InfluxDB 3 Enterprise includes features like high availability, scalability, and enhanced security, making it ideal for enterprise-scale applications that require reliability and scalability. Three recommended cluster setups are available: a two-node setup for simplicity and cost efficiency, a three-node setup for optimized performance and scalability, and a five-node+ cluster for high-throughput scaling.

Apache DataFusion is a top-level Apache Software Foundation project that has achieved best-in-class performance for querying Parquet files in the ClickBench benchmark and had a research paper accepted for SIGMOD. It provides a powerful, embeddable, and scalable solution out of the box, leveling the playing field for startups and open source projects to compete with established data platforms. DataFusion is used by various projects such as Pydantic Logfire, InfluxDB, OpenObserve, LanceDB, SpiceAI, Cube, and Arroyo to power real-time observability, AI-driven applications, vector databases, semantic layers, stream processing frameworks, and more. The project's versatility, interoperability with Apache Arrow and Parquet, Rust programming language, extensibility, and scalability make it an attractive choice for developers building data-intensive applications. With its growing open source community, strong performance benchmarks, and continued innovation, DataFusion is positioned to become the foundation of the next generation of data-driven applications.

The InfluxDB 3 Processing Engine is a Python-based VM that enables data transformation, enrichment, downsampling, alerting, and more within the database itself. The engine was built to solve real-world problems and is designed to integrate with existing workloads, providing a flexible and scalable tool for different workloads such as real-time vs batch processing. The Processing Engine conists of plugins and triggers that can be used to automate workflows, enrich data, and create custom processing logic within the database. It provides a shared API that allows for seamless interaction with the database, enabling users to write enriched/processed data back into the system. The engine also includes built-in logging functions and supports dynamic arguments, allowing users to make their plugins more flexible and adaptable to different situations.

Amazon Timestream for InfluxDB Read Replicas is a new offering that enables developers to scale time series workloads on AWS without the complexity of managing distributed systems. This feature builds on the existing partnership between InfluxData and AWS, allowing developers to run open source InfluxDB natively on AWS with full management and scalability benefits. With Amazon Timestream for InfluxDB Read Replicas, developers can create copies of a primary database instance in different availability zones to boost query performance, scalability, and reliability without operational overhead or complexity. This feature is designed for global accessibility and ease of use, providing support across 12 AWS regions and compatibility with InfluxDB 2.7 OSS. Developers can leverage Read Replicas for distributed query scaling, hot standby failover, and improved write throughput, ensuring continuous uptime and consistent performance for mission-critical workloads.

InfluxData and AWS have expanded their strategic partnership with the introduction of Amazon Timestream for InfluxDB Read Replicas, a new capability designed to deliver enterprise-grade scalability and reliability to time series workloads on AWS. This new offering provides scalable query capacity, rapid failover, and uninterrupted access to critical time series data without the complexity of cluster management, addressing the challenges enterprises face as they scale these workloads. The availability of Read Replicas makes it possible for developers to scale their mission-critical workloads with high performance and reliability, allowing them to focus on high-speed data ingestion while maintaining optimal performance and continuous operation in the event of a failover. With Amazon Timestream for InfluxDB Read Replicas, organizations can ensure the resilience of time series data and meet the growing demand for seamless, always-on operations.

This blog post discusses the author's efforts to optimize their solar-powered battery savings by adjusting its charging schedule. The author initially found that the battery was not being sufficiently charged, leading to lower daily savings rates. To address this, they configured the Solis inverter to charge from the grid during periods of low energy prices (known as "power-ups") and let it discharge at night when prices were higher. This approach allowed them to avoid negative savings days and increase their average daily savings by 21%. The author also discovered that using the battery more, especially on cloudy days, helped increase savings by about 20%. Future improvements include automating the charging schedule to avoid negative savings and ensure a sensible charge level on cloudy days. Despite still being far from breaking even, the author believes there is potential for additional savings with the current setup.

Cybersecurity in the Industrial Internet of Things (IIoT) is a pressing concern, with critical infrastructure such as energy grids and telecom networks being prime targets for cyberattacks. To stay ahead of bad actors, organizations must adopt real-time monitoring, automated threat detection, and compliance-driven security measures. InfluxDB can help by enabling efficient time series data management, enhancing visibility, and streamlining incident response, particularly in edge computing environments where data is transmitted securely and synced between edge devices and central systems. A proactive cybersecurity strategy ensures IIoT systems remain secure, resilient, and adaptive in an interconnected world. Key takeaways include securing device access, network protection, 24x7 monitoring and alerts, leveraging machine learning, conducting security audits, securing data transmission, and more.

The InfluxDB 3 Last Value Cache (LVC) is a powerful tool that allows users to cache the most recent values for specific fields in a table, improving the performance of queries that return the most recent value of a field or the last N values of a field. The LVC stores data in memory and can provide query responses in under 10ms, simplifying common query types such as retrieving the latest reported values for specific fields. The LVC is associated with a table and has its own unique name, key columns, value columns, and count. It's essential to balance the size of the LVCs with the amount of memory required to maintain them, considering factors like high cardinality key columns and value counts. When using the LVC, users can query data from it using SQL queries, excluding the `last_cache()` function in InfluxQL. The cache is flushed when the server stops, and new values are written only when data is updated.

This tutorial teaches users how to forecast time series data using machine learning (ML) and InfluxDB. The project uses the Prophet library, a popular open-source ML library developed by Facebook research, to handle seasonality and trend changes in historical weather data from London, UK. The code fetches historical weather data from the Open-Meteo API, stores it in InfluxDB using the serverless version of InfluxDB 3, reads the data from InfluxDB, and uses Prophet to forecast the temperature for the next 30 days. The tutorial highlights the merits and drawbacks of using ML models versus traditional statistical techniques like ARIMA and introduces time series-specific LLMs as a new approach to forecasting. The project demonstrates batch processing, which is suitable for long-term forecasts like weather, but notes that real-time forecasting is needed for applications like stock prices.

This blog post covers the essential command-line tools and workflows for managing and interacting with InfluxDB 3 Core and Enterprise instances. It assumes users are familiar with InfluxDB 3 Core, which is the open-source version of InfluxDB 3 Enterprise. The CLI commands in Core are identical to those in Enterprise, making it an excellent choice as an edge data collector or for learning purposes. The post provides a comprehensive guide on how to start and stop the server, create and write data to databases using authentication tokens, query data efficiently with SQL and InfluxQL, set up performance-boosting features like last value cache and meta-cache, and delete databases and tables when necessary. The blog also covers the naming conventions for buckets, measurements, fields, tags, and provides examples of how to use these commands in practice, including writing data directly or using cURL to interact with the database.