January 2023 Summaries

Crosser is a Swedish company that develops a streaming analytics platform designed for industrial IoT (IIoT) applications, aiming to provide real-time insights and innovation by integrating data from connected devices. Legacy data historians have limitations in the modern digital world, but Crosser's platform combines with InfluxDB and Grafana to offer a robust and interoperable data historian solution. The Crosser platform consists of two main components: Control Center and Edge nodes, which collect and analyze data near industrial machinery, while also containerizing InfluxDB and Grafana for storage and visualization. Deployment options vary from on-premises to cloud-based configurations, allowing users to scale their solution according to their needs and requirements. By combining Crosser with InfluxDB and Grafana, industrial operators can gain modern, digital insights into their operations in real-time.

InfluxData, a customer of CircleCI, is proactively rotating its secrets following an alert from CircleCI recommending users rotate their secrets due to a potential security exposure. The company uses CircleCI for its Open Source products and stored the Linux packaging signing key with fingerprint 05CE 1508 5FC0 9D18 E99E FB22 684A 14CF 2582 E0C5 in CircleCI, which is being rotated to a new key. InfluxData will provide updates to its repositories once the new key is rolled out, and users must configure their systems to use the new key to continue downloading and verifying software from official InfluxData repositories. The company has verified that its official repositories have not been compromised and that the software signed by the old key can be trusted.

SQL Server timestamps are a crucial component of relational databases, allowing for efficient storage and retrieval of data that changes frequently. They provide an accurate way to track changes to data over time, helping to prevent concurrency issues and ensure data integrity. Timestamps can be used to store dates and times with precision, and their format is typically YYYY-MM-DD HH:MM:SS.NNN. In a .NET application, SQL Server timestamps can be used to create a timestamp column in a database table, such as the `FirstCreatedTime` and `LastUpdatedTime` columns in the `UserProfiles` table. By using timestamps, developers can track changes to data over time, provide an audit trail of data changes, and improve performance by tracking when data was last accessed. Additionally, timestamps can be used to understand how data changes over time, making them a valuable tool for applications that rely heavily on timestamped data.

D3.js is a powerful JavaScript library used for data visualization, allowing developers to create interactive and dynamic visualizations. When combined with React, it enables developers to build complex UIs using minimal code, making it an ideal choice for creating data presentations. This guide provides a step-by-step tutorial on how to use D3.js and React to create simple bar charts, as well as a line chart example. By leveraging the flexibility and power of D3.js and React, developers can quickly create informative and engaging charts and graphs to help make complex data easier to understand. With its emphasis on web standards and powerful visualization components, D3.js is a valuable tool for anyone looking to bring data to life using HTML, SVG, and CSS.

This tutorial explains the concept of time-series data and forecasting, its importance in today's rapidly changing society, and how to use InfluxDB and Prophet to build a forecasting model. It covers the basics of time-series data, including its collection and analysis, as well as more advanced topics like predictive maintenance and energy efficiency management. The tutorial also provides step-by-step instructions on how to set up InfluxDB, load data into it, and build a forecasting model using Prophet. Additionally, it discusses the flexibility of the Prophet model, including its ability to handle outliers and variations in time-series data. Overall, this tutorial aims to provide readers with a comprehensive understanding of time-series data and forecasting, as well as practical skills for building effective forecasting models.

Apache Arrow, Apache Parquet, and InfluxDB IOx are key components of the next generation storage engine being built by InfluxData. The new storage engine uses Rust for performance reasons, but developers used pyarrow client library to interact with it. The basics of Apache Arrow include data containers such as Arrays and Tables, which can be created using code snippets like `pa.array()` and `pa.Table.from_arrays()`. The primary feature of Arrow is providing facilities for saving and restoring tabular data in formats like Parquet. A practical example uses Apache Arrow to pull a large sample dataset from InfluxDB's TSM engine, convert it into an Arrow table structure, save it as partitioned Parquet files, and then reload the partitions to perform basic aggregations on the data. The new storage engine emphasizes its commitment to the greater ecosystem by allowing exportation of Parquet files, enabling analysis in platforms like Rapid Miner.

InfluxData has been recognized with several awards for its workplace culture and commitment to diversity and inclusion. The company has won awards such as Best Company Culture, Best CEOs, and Best Small and Mid-Sized Companies for Women, reflecting the positive sentiment of its employees and validation of its mission to deliver on core values. InfluxData's embrace of remote work has also contributed to its success, allowing it to build a diverse and inclusive workforce while maintaining productivity and work-life balance. The company prioritizes its core values of valuing each other, getting stuff done, humility-driven learning, embracing failure, and commitment to open source, which are reflected in its award-winning culture.

Docker introduces complexities that need to be managed, such as infrastructure and application monitoring. Traditional monitoring solutions may not be suitable for Docker-based workloads due to the abstraction offered by containers. InfluxDB and Telegraf can help mitigate these complexities by providing tools for monitoring and collecting metrics from a Docker installation. By setting up and configuring InfluxDB and Telegraf as Docker containers, users can collect critical metrics such as CPU and memory usage, disk and network I/O, and visualize the data in InfluxDB's Data Explorer page. This enables proactive problem-solving, resource allocation optimizations, and early detection of performance issues, ultimately improving application reliability and customer satisfaction.

Apache Arrow is an open-source framework that enables efficient in-memory columnar data representation, facilitating interoperability among various processing engines. It was developed by several open-source leaders, including Wes McKinney, creator of Pandas, to solve the problem of making Pandas interoperable with data processing systems. Apache Arrow provides zero-copy reads, reducing memory requirements and CPU cycles, and is designed for modern CPUs and GPUs to process data in parallel. Several companies, including InfluxDB, use Apache Arrow as a critical component in their architecture, such as converting Pandas DataFrames to Spark DataFrames or storing time series data efficiently. The framework has been widely adopted due to its efficient columnar memory exchange, and its builders have made significant contributions to other open-source projects, including Fast, memory-efficient sorts, performance improvements, and making the Arrow crate safe by default.

Apache Arrow is an open-source project that aims to provide a standardized columnar memory format for flat and hierarchical data, making analytics workloads more efficient for modern CPU and GPU hardware. It solves the problem of performance overhead involved with moving data between different tools and systems as part of data processing pipelines by creating a common standard for transferring and manipulating large amounts of data efficiently. By adopting Arrow, developers can experience significant performance gains due to its column-based format, which is designed for modern CPUs and GPUs, allowing for parallel processing and reducing memory requirements. Additionally, Arrow integrates well with other projects like Apache Parquet, making it easier to manage the life cycle and movement of data between systems. The project has gained major adoption and features a growing ecosystem of tools and languages that can use the Arrow format, making it a lingua franca for data transfer and manipulation.

A column database is an alternative to traditional relational databases, storing data organized into columns instead of rows on disk. The main advantage of a columnar database is that it can significantly reduce the amount of disk space required to store the data due to improved compression ratios. This results in faster processing times for analytic-type queries compared to traditional row-based databases. Column databases are optimized for analytics workloads and provide orders of magnitude better performance, thanks to factors such as improved compression, storing multiple versions of the same column, adaptive indexing, vectorized processing, and optimized joining of columns. However, they come with trade-offs, including slower update and read query performance compared to traditional relational databases. Column databases are suitable for applications that require large amounts of data analysis, such as business intelligence, application performance monitoring, IoT, and time series data management. They can be used in conjunction with other technologies like Apache Arrow, DataFusion, and Parquet to enhance performance and compatibility.