May 2021 Summaries

This blog post discusses how to monitor your InfluxDB Open Source Instances with InfluxDB Cloud. It provides an architecture overview for deploying a Telegraf sidecar that sits next to your InfluxDB instance, scraping Prometheus metrics from each instance and pushing them to a centralized InfluxDB Cloud account. The post also covers setting up InfluxDB OSS, deploying and configuring Telegraf, combining the information in InfluxDB Cloud, adding a deadman alert, and getting notified. It concludes by emphasizing that this setup can be customized to monitor for other important events and provides an easy way to add monitoring to on-prem deployments of InfluxDB OSS instances.

InfluxDB has released maintenance updates for both its open-source version (InfluxDB OSS 1.8.6) and enterprise version (InfluxDB Enterprise 1.8.6). While there are no specific improvements in the OSS edition, the Enterprise release includes fixes for a critical security issue related to fine-grained authorization feature, logging improvements, and other defect fixes. The new versions aim to enhance user experience and ensure data security.

Telegraf 1.18.3 has been released as a maintenance update with several improvements. These include adding FreeBSD armv7 build, migrating dependencies, updating hashicorp/consul/api module to v1.8.1, and more. The new release also includes the ability to set user agent when scraping Prometheus metrics, an issue fix for Kinesis Consumer Input, and enabling gzip compression for Elasticsearch Output. The latest open source release binaries can be found on InfluxData's downloads page.

The article discusses how to consolidate multiple OSS instances into one cloud account using Flux's to() or experimental.to() functions. It also provides information on building alerts based on the data collected in remote instances and improving system resiliency by configuring a Telegraf instance that proxies the cloud account. The article includes links to related blog posts and resources for further reading.

The article discusses the design of a Parquet Catalog for InfluxDB IOx, a new in-memory columnar database that uses object storage for persistence. It explains why existing catalog standards like Apache Hive, Delta Lake, and Apache Iceberg were not suitable for their needs and how they decided to implement their own design. The catalog is focused on tracking what exists in object storage and efficiently keeping track of schema and statistics information for the Parquet files that InfluxDB IOx writes to object storage. It also supports soft deletes, allowing users to delete data but have it still be around for some period if needed. The design borrows many concepts from these three projects and uses the Parquet metadata format in Apache Thrift to keep information about metadata and statistics.

InfluxDB OSS 2.x has seen a steady upgrade rate since its initial release in November 2020, with more than 10% of the community successfully upgrading. The upcoming InfluxDB OSS 2.1 will bring significant Flux performance improvements and feature enhancements such as collaboration features. For InfluxDB Enterprise customers, the focus is on ensuring compatibility with new data access & processing capabilities while maintaining the current deployment architecture. InfluxDB 1.8 will continue to be maintained and receive defect fixes through the end of 2021, but no further official releases are planned for the 1.x line.

This article discusses three ways to keep cardinality under control when using Telegraf, an open-source plugin-driver metrics collection agent of InfluxDB. The first method involves adjusting the data collection interval based on specific needs and policies. The second method is filtering metrics to collect only the necessary data. Lastly, retention policy and downsampling can be customized to control cardinality by setting appropriate data life duration in accordance with compliance requirements or internal policies.

Index Exchange, a global marketplace for digital media advertising, has modernized its DevOps practices by implementing the InfluxDB platform. The company previously relied on email alerts for monitoring infrastructure data but decided to rethink their approach and explore other technologies. They chose InfluxDB as it could handle longer-term storage of metrics best. Index Exchange uses InfluxDB to store real-time data, set up alerts about the data store in the InfluxDB cluster, and collect various metrics including CPU memory and error types. The company also uses Kapacitor for alerting and data computations, as well as Chronograf and Grafana for visualization purposes.

The article discusses the use of ConfigCat, a hosted service for feature flag and configuration management, at InfluxData. ConfigCat enables users to control features and target specific user segments based on various attributes. It provides SDKs for different languages, but Elixir was not initially supported. InfluxData developed an internal Elixir SDK and later collaborated with another company, Drover, to open source it and make it available to the public through Elixir's hex.pm package registry. The collaboration resulted in a more consistent and feature-rich Elixir SDK for ConfigCat.