November 2019 Summaries

AWS re:Invent 2019 is a leading conference for Amazon Web Services users, customers, and partners, featuring over 2500 technical sessions, training, certification, partner exhibits, parties, and more. InfluxData will be participating at the event, with its VP of Product presenting on how InfluxDB works with AWS for DevOps monitoring and IoT sensor data collection. The presentation aims to improve time-to-solve problems while reducing code. InfluxDB provides various integration points into AWS, including CloudWatch, EC2, ECS, EKS, Kinesis, RDS, synthetic monitoring, and Programmatic Billing. Additionally, the company will have a booth at re:Invent, offering assistance with monitoring systems and applications built on AWS.

InfluxData has signed major new customers in 2019, including Visa Inc., NASA, Airbus, Renault, and Yoox, highlighting the extensive demand for time series technology. InfluxDB is a leading time series platform offering real-time observability, management of time-sensitive workloads, monitoring system, dashboarding engine, powerful analytics capabilities, and an event and metrics processor. Its fast ingest, elastic scalability, and compression enable efficient storage and analysis of large volumes of time-stamped data at a lower cost than relational or other NoSQL databases. The company's new customers span across industries and size, from Fortune 50 to stealth-mode tech companies, with deployments in various sectors such as automotive, manufacturing, software, telecommunications, retail, and government.

Telegraf is an agent that runs on the operating system of choice, gathering metrics and events from various sources and sending them to one or more sinks such as InfluxDB or Kafka. It supports multiple inputs, including over 200 plugins for retrieving information from applications, hardware, and software. To run Telegraf on Windows, download the Windows binaries from the official website, extract the archive into the Program Files folder, create a conf subdirectory, and copy the telegraf.conf as conf\inputs.conf. The outputs section of the file is then configured to send data to InfluxDB Cloud specifically, using credentials such as a token, organization name, and bucket name. To install Telegraf as a Windows service, run the command .\telegraf --service install --config-directory 'C:\Program Files\telegraf\conf'. The Telegraf service can be started with the command net start. To use an input plugin, such as the Docker input plugin, enable it in the telegraf.conf file by copying configuration code from the GitHub repository into the inputs.conf file. This allows users to switch on the plugin without installing it, as it is already part of the product.

Kubernetes has become a crucial tool in DevOps for scaling operations of containerized microservice architecture. However, managing deployment, scaling, upgrading, and updating of these services is not trivial, making automation essential. InfluxDB Cloud 2.0 adopted Kubernetes as part of its microservices architecture, but monitoring became an issue due to the ever-growing number of endpoints exposed by SREs and developers. The answer lies in sidecar deployment, where a scraper is contained within a pod, isolating its impact on other workloads. This approach solved the scaling issue without adding complexity or burden to operations. Additionally, Telegraf, a lightweight metrics collection agent deployed as a sidecar, monitors all exposed metrics by the application, service, or microservice, ensuring no impact on IT Operations or developers' needs for metrics. Furthermore, Telegraf's self-watch feature allows it to monitor its own performance and detect missing metrics, while its ability to limit tags and configure runtime settings empowers developers with observability and predictability. The key lesson learned is that scaling monitoring should not add complexity but rather empower developers with tools like Telegraf, ensuring monitoring does its job without burdening operations.

A new release of InfluxDB 2.0 Alpha is now available, which includes various enhancements such as upgrades to Clockface 1.0.6 for UI bug fixes and improvements, a new feature allowing Telegraf users to quickly view and download the InfluxDB v2 Output Plugin configuration, support for displaying String fields in Single Stat graphs, and an updated Flux library version 0.54.0. This release is not intended for production usage but rather for providing feedback on functionality, user experience, and APIs.

Monitoring Windows Services with Grafana, InfluxDB and Telegraf is a tutorial that guides DevOps engineers on how to set up a complete stack for monitoring Windows services using state-of-the-art tools like Grafana v6.1.4, InfluxDB v1.7.3, and Telegraf 1.10.3. The goal is to build an efficient way to monitor applications by leveraging the Performance Counters API on Windows machines, which provides native components for recording and monitoring data for various tools such as CPU, disks, processes, databases, and ASP.NET applications. The tutorial covers installing the tools, configuring Telegraf to query the Performance Counters API, building a dashboard with Grafana, and setting up alerts to notify DevOps teams when services fail. By following this tutorial, users can gain valuable insights into their system's performance and take proactive measures to prevent costly downtime.

Telegraf 1.12.6 is a new maintenance release that includes improvements to several plugins, such as Agent, Docker Log Input, Ping Input, and MongoDB Input, addressing issues related to plugin errors, precision, and panic handling. The latest open source release can be found on the InfluxData downloads page, with additional updates available in other products like Chronograf and Kapacitor.

InfluxData's integration with PagerDuty enables organizations to adopt AIOps (Artificial Intelligence for IT Operations) by providing real-time full-stack visibility, intelligent alerting, and response automation. This integration allows for the analysis of historic data to be adaptive, handle multidimensional risk scenarios, and event alerting, which is particularly important in Kubernetes environments due to their ephemeral nature. Real-time monitoring, intelligent alerting, and orchestration of the appropriate response for each event are key to excellence in operations and user experience, with the right solutions integrated to take full advantage of data-driven workflows on both operational and business levels.

Telegraf is a software agent for sending time-series data to InfluxDB, an open-source time series database. It has hundreds of plugins to collect different types of time-series data. The http_response plugin sends HTTP/HTTPS requests to a list of URLs and sends metrics about the responses back to InfluxDB. To set up synthetic monitoring of AWS API endpoints using Telegraf and InfluxDB Cloud, one needs to create a configuration file that specifies the organization, bucket, and URLs to monitor. The configuration file also includes settings such as follow_redirects and response_timeout. A list of all AWS service endpoints can be compiled from the AWS documentation page or by running a command in Terminal. Monitoring AWS is necessary because AWS has its own Service Health Dashboard, which may not always provide accurate information about the availability of services. By using Telegraf and InfluxDB Cloud, developers can monitor their applications' performance and ensure that they are working as expected.

Google Cloud Next UK 2019 is an annual conference that brings together Google Cloud customers, developers, and partners to learn about and build next-generation cloud-native applications, infrastructure, security, and smart analytics. InfluxData will participate in the event, showcasing its partnership with Google to help monitor applications, microservices, and infrastructure using its time series data platform deeply integrated with Google Cloud. The company will provide insights into how customers like Walmart use InfluxData to monitor their Kubernetes infrastructure, discuss integration points between InfluxDB and Google Cloud services such as BigQuery and Stackdriver, and offer free swag including socks and stickers to attendees at its booth.

InfluxData will be participating in Kubecon/CloudNativeCon North America 2019 at the San Diego Convention Center, showcasing how InfluxDB can help monitor Kubernetes infrastructure, addressing a key challenge faced by Kubernetes users. The company will demonstrate its single, integrated time series data platform that includes data capture, processing, storage, visualization, and analysis, providing a solution to simplify Kubernetes monitoring. InfluxData will also be giving away InfluxDB socks and stickers at their booth, number G13.

Rick Brown, a developer, created a Telegraf gateway to efficiently send data from his home network to remote instances of InfluxDB Cloud. He wanted to visualize all devices in his network, monitor performance, and automate tasks without having to edit multiple configuration files for each device. To achieve this, he installed a container for Telegraf, enabled UDP and HTTP inputs, and configured outputs for both local and cloud-based InfluxDB instances. The gateway collates data from various input sources, including Telegraf, HTTP custom endpoints, and UDP, making administration easier. By implementing a Telegraf gateway, Brown simplifies his workflow and reduces the need for manual configuration updates.

Houghton Mifflin Harcourt, a nearly 200-year-old publishing company, faced challenges in gaining real-time visibility into their AWS spend, optimizing DevOps monitoring, and gathering performance metrics for its online educational business. To address these challenges, the company chose InfluxDB Enterprise, an enterprise-grade time series platform, to handle large cardinality and provide high availability, horizontal scalability, and LDAP and OAuth support. Additionally, they deployed Telegraf, a metrics collection agent, to optimize spend and performance, and Kapacitor, a native data processing engine, to process stream and batch data from InfluxDB in real-time. With InfluxDB, Houghton Mifflin Harcourt gained real-time visibility into its AWS spend, optimized DevOps monitoring, and gathered performance metrics, enabling the company to make data-driven decisions and improve its operations.

InfluxDB is a time-series database that is better than relational databases like Postgres and document databases like MongoDB at collecting time series data. Time series data is information for which you want to track the history of change over time, such as temperature, CPU utilization, or stock prices. Telegraf is an agent platform that collects time-series data and sends it to InfluxDB, which then stores the data for later analysis and presentation. To get started with InfluxDB and Telegraf on a Mac, you need to sign up for a free account of InfluxDB Cloud 2.0, load data into InfluxDB using Telegraf, create a configuration, and install Telegraf using Homebrew. You also need to set the INFLUX_TOKEN system variable and copy/paste the telegraf command along with your unique connection URL. Once you have done this, Telegraf starts monitoring various system metrics, and you can see a chart of the data in the Data Explorer.

InfluxDB Cloud is now available in Europe, specifically on AWS EU (Frankfurt) region, offering benefits such as reduced latency, lower costs, and increased redundancy for users who require data residency laws or have applications already running on the same region. The service also includes new features like usage-based pricing, secure secret store, developer-friendly client libraries, and additional dashboard templates to support low-code workflow automations with Pipedream.

InfluxData has launched its serverless time series platform-as-a-service, InfluxDB Cloud 2.0, in Europe, aiming to support enterprise transformation with time series technology. The platform is designed to handle the massive volumes of time-stamped data produced by IoT devices and applications, providing real-time data insights and empowering businesses to build transformative monitoring, analytics, and IoT applications. With its rapid ingestion rate, highly efficient compute, and advanced analytic capabilities, InfluxDB Cloud 2.0 is built to meet the specialized requirements of today's time-sensitive workloads, supporting industries such as DevOps initiatives and Industry 4.0 projects. The platform also includes a free rate-limited tier, transparent usage-based pricing, and a new data scripting language called Flux, which enables developers to extract valuable insights from data in real-time.

A new maintenance release for Telegraf version 1.12.5 is available now, addressing various plugin improvements including Ping Input, SQL Server Input, Cloudwatch Input, MongoDB, Jenkins, Docker, MySQL and several other plugins, with fixes for issues such as missing ping errors, server properties query, debug level messages, metric creation when a node is offline, uptime_ns calculation, field type conflicts and more.

InfluxDB is a time series database used for DevOps monitoring and dashboarding, built in 2013 by InfluxData, and widely deployed in distributed and multicloud environments. Docker can serve as a good fit for InfluxDB due to its virtualization environment that provides an easy way to create, manage, and delete containers on the fly. To install InfluxDB on Docker, one can either prepare their filesystem manually or use scripts to automate the initialization process. The official InfluxDB image is part of the Official Docker Images, which includes Telegraf, Chronograf, and Kapacitor as other tools of the TICK Stack. To run an InfluxDB container, one needs to map volumes from their local filesystem to the container, storing configuration files and data on the host system. The InfluxDB container can be started with the command `docker run -d -p 8086:8086 --user 997:997 --name=influxdb -v /etc/influxdb/influxdb.conf:/etc/influxdb/influxdb.conf -v /var/lib/influxdb:/var/lib/influxdb influxdb config`. After starting the container, one can verify that InfluxDB is correctly running by checking the HTTP API and verifying authentication. Additionally, enabling authentication on InfluxDB requires creating an administrator account using the `influx` command inside the container, followed by updating the configuration file to enable HTTP authentication.

InfluxData's 2019 engineering internship program aimed to integrate interns into the team processes, providing them with hands-on experience and guidance from experienced engineers. The company received valuable advice from former interns on how to improve the program, including the importance of asking questions early on, sharing experiences with colleagues, learning to work independently, and being proactive in seeking help when needed. These insights highlight the challenges faced by interns and provide actionable tips for future interns and organizations looking to create similar programs.

IoT World Today | IoT Sensing and the Importance of a Time-Series DatabaseBy Brian Buntz` In 2016, when a sinkhole opened up near the Ponte Vecchio bridge in Florence (Italy), authorities worried the famed structure could be at risk. This is when the IIoT company Worldsensing stepped in to enlist InfluxData's help to monitor the bridge. Worldsensing used wireless sensors to determine the bridge was safe, while using a time series database from InfluxData on the backend to look for anomalies in the sensor data. The use of a time-series database enabled the efficient analysis and storage of large amounts of IoT sensor data, allowing for early detection of any potential issues with the bridge's structural integrity.

InfluxData's remote internship program was a success, with 5 interns hired in 2019, and the company views interns as open source community contributors, internal team members, and students, providing guidance and mentorship to help them grow. To succeed with remote interns, InfluxData found that preventing isolation is key, hiring interns in cohorts of at least two, assigning individual mentors, and developing new managers who can lead a separate planning/scrum meeting every day. Team work principles include not wasting too much time on training, assigning large epics filled with small tasks to the group, doing daily check-ins, having code reviews with two reviewers, and occasionally doing live reviews. The company also found that having a mid-season gathering at a central location was highly successful in team-building and friendship-building.

InfluxData's engineering internships program was launched in 2019 with a large class of 5 interns hired into storage and query language teams. The company aimed to develop an entry-level hiring pipeline by infusing energy and new ideas into its teams through mentorship, creating a culture of learning and growth. With the goal of getting "stuff done," InfluxData hires interns to work on products, and with patience, training, and mentorship, 100% of their interns successfully merged pull requests into production. The program is built around core values such as humility driving learning, embracing failure, and committing to open-source contributions. The outcomes of the program have exceeded expectations, with full-time engineers gaining valuable mentoring experience and interns learning new skills while making productive contributions to the company's code base.

InfluxDB 1.7.9 is a maintenance release that includes several key fixes for the open-source OSS version of InfluxDB, including updates to Golang and InfluxQL dependencies, improvements to rate limiting and wait functionality, and enhancements to authentication options. The release addresses security vulnerabilities and provides stability and performance improvements.