August 2022 Summaries

InfluxData developed a simulator called Gems to replicate the functionality of AWS Marketplace for testing purposes, eliminating the need for third-party resources in their original application. The Gems simulator mimics key services like REST API, Simple Notification Service (SNS), and Simple Queue Service (SQS) to ensure a similar user onboarding flow while testing with Quartz, without relying on AWS's infrastructure. This allows InfluxData to test their SaaS product more thoroughly and efficiently, preparing them for future integrations with other cloud services like GCP and Azure.

This tutorial demonstrates how to use Python to retrieve hourly weather data from the OpenWeatherMap API, convert it to a Pandas DataFrame, and write it to InfluxDB, a time-series database. The author uses the InfluxDB Python Client library to simplify the process of writing data to InfluxDB, which is designed for handling high ingest requirements and large amounts of time-series data. By following these steps, users can integrate their weather data into InfluxDB, enabling them to query and analyze their data using various tools and features provided by InfluxDB.

Ekopak, a Belgian company specializing in water treatment installations for industrial purposes, has implemented a more efficient data management system using Factry Historian and InfluxDB. This new system allows them to analyze large amounts of data automatically, visualize it through dashboards, and make proactive maintenance decisions to prevent equipment failures. By transitioning from manual data analysis and Excel files to an automated system, Ekopak is able to save time, reduce errors, and improve its ability to train new employees quickly. The use of InfluxDB enables the company to store and manage vast amounts of industrial data, providing valuable insights into machinery performance and water treatment processes.

InfluxDB Cloud offers a powerful feature called "Deadman Check" that allows users to set up automated tasks to monitor for sudden shocks or delays in the flow of incoming data, and can alert them through third-party messaging platforms. To create this check, users navigate to the "Alerts" section in InfluxDB Cloud 2.0, select the "Deadman Check" type, define the query and configure the settings, including scheduling, status message, and deadman threshold. Users then need to configure a Slack App as an endpoint and create a notification endpoint, before creating a notification rule that triggers when the check detects a deadman scenario. This feature is particularly useful for monitoring large amounts of historical data or smart home devices, and can help prevent service outages from causing significant disruptions.

InfluxDB OSS 2.4 has been released, bringing multiple new features, security updates, and bug fixes. The new version includes improved user model and CLI additions, such as operator instance-level permissions without an operator token and the influx remote command. Flux enhancements have also been added, including new join functions, promoted experimental functions to stable packages, and support for conditional expressions in vectorized map(). Security updates address issues with YAML parsing and storage authorization, while bug fixes resolve race conditions and errors related to replication and authorization. Additionally, the release includes cumulative security fixes for Go versions 1.17.8-1.18.3.

InfluxDB is a purpose-built time series platform used by Pinnacle 21 to streamline data pipelines for faster and more accurate clinical trial data. Pinnacle 21, a software company specializing in life sciences solutions, uses InfluxDB's open source version to replace their previous monitoring solution, Datadog. Josh Gitlin, Director of DevOps at Pinnacle 21, shares how they implemented Telegraf as the new monitoring tool, leveraging its power and flexibility to collect and monitor APM metrics, logs, and KPIs. The team also uses InfluxDB Cloud for redundancy and scalability, and Grafana for visualization and alerts. Josh emphasizes the importance of using Flux, a powerful language for InfluxDB, and offers tips and tricks for anyone starting their monitoring journey with InfluxDB.

Efficient manufacturing is crucial for saving companies time, money, and energy. Overall Equipment Effectiveness (OEE) measures efficiency as a percentage, aiming for perfect quality, high speed, and always-running machinery. By tracking OEE over time, manufacturers can identify bottlenecks, improve maintenance, and optimize processes. Availability measures the frequency of machine downtime, Performance assesses real-world manufacturing speed compared to ideal speeds, and Quality evaluates product defect rates. Calculating OEE requires handling large volumes of time series data from sensors on equipment, and companies must manage this data strategically to gain meaningful insights into their efficiency.

Telegraf is an open source plugin-based agent that gathers data from various sources and sends it to a database, allowing users to transform data before sending it to its endpoint. Getting started with Telegraf can be challenging, but resources like InfluxDB University's free course offer guidance on common connection issues and troubleshooting tips. To resolve problems, users should check logs, try running Telegraf with specific flags, verify their API token and input connections, and review the configuration file for errors or mistakes in TOML format. With proper setup and practice, users can become proficient in using Telegraf to collect and transform data.

InfluxData has announced new serverless capabilities for its cloud-based time series platform InfluxDB Cloud, enabling faster insights by expediting device to cloud data transfers with MQTT. This allows developers to subscribe to, process, transform, and store real-time data from messaging and other public and private brokers and queues without additional software or code. The Native Collectors introduce the fastest way to get data from third-party brokers into InfluxDB Cloud, simplifying time series data pipelines and applications alike. This update aims to support organizations' growing need for high-quality data governance and analytics in cloud environments, reducing complexity through platform consolidation and delivering real-time data ingestion with unprecedented simplicity, speed, and scale.

InfluxData has announced the immediate availability of InfluxDB Native Collectors, a new suite of capabilities that enable developers to subscribe directly to supported message brokers, making it easy to collect, process, and analyze data. This feature allows for faster cloud-to-cloud data transfers, enabling developers to focus on building and scaling applications with their time series data. With three configuration steps, developers can start collecting and analyzing data from messaging brokers without the need for additional software or new code. The introduction of InfluxDB Native Collectors adds a low-code setup option, supporting popular cloud subscriptions, brokers, and services, including MQTT, Kafka, AMQP, and more. This feature aims to optimize workflows for broker-based data sources, providing conveniences such as subscription wildcards, dynamic data routing, and JSON string parsing. With InfluxDB Native Collectors, developers can realize benefits without a high cost of adoption or evaluation, making it easier to evaluate the platform's value proposition.

Here's a neutral and interesting summary of the text: The 2022 Intern Hackathon at InfluxDB brought together eleven interns from various departments to brainstorm ideas for projects that could be completed in one week. Two teams were formed, with the author working on a stock market sentiment bot. The team utilized a Google Doc to organize their tasks, GitHub to manage the project code, and InfluxDB's Python Client library to store and query data. Despite facing challenges such as estimating market behavior and architecting machine learning models for time-series databases, the team successfully deployed several concepts, including web scraping, data cleaning, and model predictions. The project was completed in one week, demonstrating the interns' ability to learn new technologies and build a relatively complete program that can be used in larger-scale applications.

InfluxData has donated a generic object store implementation to the Apache Arrow project, allowing developers to easily interact with various cloud storage services such as AWS S3, Azure Blob Storage, Google Cloud Storage, and local files, by simply configuring runtime settings. This donation aims to accelerate innovation in the Rust ecosystem, particularly for building cloud-agnostic services that handle user-uploaded content or high-performance analytics systems. The object store crate provides a flexible and extensible solution, enabling developers to focus on their system's logic while abstracting away low-level details of interacting with different object storage implementations. With this donation, InfluxData hopes to further accelerate the creation of high-quality analytic systems in Rust, aligning with Apache Arrow's capabilities for efficiently reading arrow-compatible files from local or remote object storage.

InfluxDB has released a new CLI onboarding wizard that simplifies the learning experience for using the command line interface. This feature eliminates the need to spend hours researching documentation, allowing users to get up and running quickly with InfluxDB. The CLI offers advantages such as reduced resources usage, high precision, repetitive-task friendly capabilities, and powerful commands and processes, making it a valuable tool for both average users and experts. To get started, sign up for a free InfluxDB Cloud account and follow the provided step-by-step guide to onboard with the InfluxDB CLI.

InfluxDB is an open-source time series database designed to handle large volumes of time-stamped data from various sources. The Python client library for InfluxDB provides a powerful way to interact with the database, allowing users to write, query, and manage data using Flux, its functional data scripting language. The library supports various APIs, including WriteApi, QueryApi, DeleteApi, TasksApi, InvokableScriptsApi, and InfluxDB administration APIs. Users can connect to InfluxDB using a client library that requires connection information, such as URL, access token, organization, and other parameters. The client library allows users to prepare data in several ways, including string format, dictionary format, Point class, Pandas DataFrame, Data Class, and Named Tuple. The library also supports asynchronous writes and provides various options for customizing the write process.

InfluxDB CLI, a standalone open source binary compatible with both OSS and InfluxDB Cloud, has released version 2.4 with enhancements for each platform. The new features include setting membership type using influx org members add command, reintroduction of the InfluxQL Shell (REPL) from v1 with an improved experience, support for calling Invokable Scripts in the cloud, and username and password authentication as alternatives to token-based authentication for OSS. Additionally, v1 dbrp names are now supported as bucket IDs in replications API, making it easier to create replications and supporting replication to an Enterprise cluster's v2/write endpoint.

InfluxDB OSS 2.4 has been released with improvements to its user model, Flux enhancements, and CLI additions. The version upgrade includes the addition of a v1 InfluxQL shell/REPL, support for v1 dbrp names as bucket IDs in replications API, and changes to the operator model. Flux has been upgraded from 0.171.0 to 0.179.0 with new join functions, built-in time function, promoted array and http.requests functions, and support for conditional expressions, constants, and literals in vectorized map().

InfluxDB is a time series database solution used by Infosys, a global IT leader, to handle data challenges such as proactive anomaly detection, triaging incidents, and maintaining high availability and SLAs. The company chose InfluxDB due to its industry recognition, scalability, analytics capabilities, and IoT solutions. Its power comes from elastic scaling and sophisticated operations like pipe forwarders, offering more granularity than alternative databases. InfluxDB provides a wealth of resources around APIs and metrics, making it seamless for development in CI/CD pipelines. It also offers group functions that aggregate along series and create new group keys based on designated properties. The Flux language allows users to retrieve data, filter it, process it, and shape it into results, with a vast library of input plugins and a "plug and play" architecture. InfluxDB helps with proactive monitoring, automation, remediation of data errors, and aligning on KPIs across multiple services. Its visualization options within the UI allow users to create dashboards that display data in various formats. The company plans to continue developing DevOps within Kubernetes using InfluxDB to address security events, application errors, and overlaying applications.

InfluxDB is a time-series database designed specifically for storing and managing large amounts of data that change over time. It provides features such as scalability, high performance, and advanced data retention options. The article discusses the importance of time-series data in various industries, including finance, healthcare, and weather forecasting. InfluxDB is highlighted as an ideal solution for storing and analyzing time-series data due to its ability to handle large amounts of data efficiently. The article also explores how to use InfluxDB with Python and TensorFlow to build a time-series forecasting model, which can be used to predict future values in a time series dataset. This involves setting up the InfluxDB account, installing the InfluxDB Python library, connecting to the database, inserting data, and building a machine learning model using TensorFlow to make predictions. The article concludes by providing additional resources for those interested in learning more about time-series databases and machine learning with InfluxDB and TensorFlow.

Telegraf has released version 1.23.4, which includes updates to its input plugins, including improvements to the AWS CloudWatch Input plugin, the Kubernetes Input plugin, and the MongoDB Input plugin. The output plugins have also been updated, with a new Coralogix dialect added for OpenTelemetry output, and support added for AWS Timestream Output. Additionally, Telegraf maintainers are seeking volunteers to test bug fixes and features in upcoming releases.

InfluxDB has released a new feature for its Tasks product, allowing users to invoke scripts and pass variables to these scripts, enabling the management, reuse, and scaling of code and processes. This feature simplifies the process of managing tasks at scale by allowing developers to create reusable scripts that can be referenced by multiple tasks with varying parameters. The power of Flux is now democratized across an organization, making it possible for non-Flux experts to benefit from complex scripts written by fluent users. To get started, users can refer to InfluxDB's documentation and sample code, or reach out to the support team for specific design or implementation questions.

InfluxDB is a time series platform used by Olympus Controls to monitor and analyze sensor metrics from machines and production in their industrial manufacturing plants. The company uses InfluxDB to collect data on robot health, including motor health, to predict maintenance windows and reduce costs. Olympus Controls implemented a solution using Telit DeviceWISE edge platform, MQTT broker, Telegraf plugin, and InfluxDB to automate the monitoring of robots and factories. Nick Armenta, Automation Engineer at Olympus Controls, found InfluxDB easy to use due to its quick querying capabilities and visualization tool, which helped him visualize data in real-time and make predictions about maintenance needs. The company used MQTT broker for load balancing and scalability, and Nick overcame pushback from his organization by educating them on the benefits of using InfluxDB for predictive maintenance.

OpenTelemetry is an open-source collection of tools, APIs, SDKs, and specifications that standardize how telemetry data is modeled and collected. It was formed from the merger of OpenCensus and OpenTracing projects and has backing from major tech companies such as Microsoft, Red Hat, Google, Amazon, Shopify, Datadog, and Facebook. The tool helps solve problems related to observability in cloud-native and microservice architectures by providing a standardized way to collect metrics, logs, and traces, making it easier for developers to debug and resolve issues, identify bottlenecks, and improve security. OpenTelemetry provides real value to applications by decoupling collection from storage, streamlining implementation, and offering a robust ecosystem and community. Its use cases include software monitoring, optimization, security, and developer productivity, while its data is stored in time series databases or other specialized databases to optimize scalability, query performance, and cost savings. By analyzing and acting on observability data, developers can automate deployment pipelines, make informed decisions, and improve overall software development and deployment processes.

InfluxDB offers a popular Python client library that allows developers to work with time series data in their preferred programming language. The library provides efficient batching capabilities, enabling users to send large amounts of data to InfluxDB at once. It also supports querying data using the Flux query language, returning results in various formats such as strings, CSVs, streams, and Pandas DataFrames. Additionally, the Python client library offers administrative APIs for tasks like creating organizations, buckets, and authorization tokens, making it easier to automate management of multiple InfluxDB instances.

In a bid to combat climate change, commercial buildings must prioritize energy efficiency. EnerKey, a Finnish company, uses data analytics to help clients optimize their energy use in their properties. By connecting utility data with weather and geospatial information, EnerKey enables clients to make informed decisions about energy consumption. The company has over 80 integrations with IoT sensors and stores large volumes of time series data using InfluxDB, a platform that can handle automation and provide support for various use cases. With the help of InfluxDB, EnerKey's clients have achieved significant energy savings, including Kesko, which is already saving 5 million Euros per year and aims to save at least 7.5% more energy by 2025.

In tech sales, there is a perception problem that deters many candidates from pursuing careers in sales, especially women and minorities. The traditional alpha-male persona is often emphasized over soft skills, which can lead to a culture of success for some but not others. However, substance over charisma is key, with skills such as discipline, intellectual curiosity, empathy, ability to navigate complexity, creativity, and problem-solving being more important than personality traits. The role of sales has shifted from pushing products to enabling customers to make informed decisions through enablement, which includes access to resources, volume discounts, and navigating security. Companies need to create a culture of success for all salespeople regardless of background, and the best sales teams are those that focus on making customer projects successful, rather than just selling. The shift to product-led growth has also changed the buying process, with customers wanting to experience technology in small doses and pay only for what they consume. Sales leaders need to demystify the sales process by automating it with technology, creating enablement plans, and hiring salespeople who care about the product and can execute against a GTM plan.

The Flux query team has developed a new join package to improve upon existing solutions and provide support for outer joins, which was a frequently requested feature. The new package provides six functions, including `tables()`, `inner()`, `full()`, `left()`, `right()`, and `time()`, with the latter three being aliases for the first three. It allows users to define left and right input table streams and compare records from these tables based on a function that takes two records as arguments, following a specific format. The package also supports different join methods, including inner, left, right, and full joins, each with its own set of rules for constructing the output record. The new package aims to improve performance by reducing row comparisons and provide flexibility in join predicates, which can be used to leverage complex comparisons in the future.

Cloud-native technology is vital for building fast, agile, scalable, and stable solutions in cloud data centers, which handle 94 percent of all workloads. Cloud-native computing provides the foundation for designing and managing applications in a modern, dynamic, and multicloud environment. It leverages cloud services to enable dynamic and agile application development while taking a modular approach to building, running, and updating software across various cloud-based services. The pillars of cloud-native computing include cloud infrastructure, modern design, microservices, containers, backing services, and automation. Cloud-native applications are characterized by disposable infrastructure, self-contained code, scalability, value-added cloud services, polyglot clouds, full-stack teams, and benefits such as faster time to market, reduced vendor lock-in, improved cost-efficiency, greater business resilience, richer customer experience, and room for growth. However, there are also limitations, including the need to redesign or create native applications, potential complexity in deploying microservices, and operational complexities due to decentralization. Overall, cloud-native development offers strategic benefits for organizations looking to leverage growth-oriented development and adapt to ongoing trends.