December 2016 Summaries

OpenStack is an open-source project that enables enterprises to deploy private clouds, with companies like PayPal and eBay using it for mission-critical services. Due to the complexity and scale of OpenStack environments, continuous logging and monitoring are crucial to preemptively address performance issues. The ELK Stack, comprising Elasticsearch, Logstash, and Kibana, is highlighted as an effective logging platform for this purpose, offering scalability and ease of use. The text provides a comprehensive guide for monitoring OpenStack using ELK, detailing how to retrieve logs via OpenStack's APIs and CLI, and emphasizes the need for authentication tokens. Monitoring is broken down into layers, including hypervisor, server, and tenant metrics, with the importance of RabbitMQ, OpenStack’s default messaging system, also noted. The guide includes configurations for using Logstash and Amazon S3 for log aggregation and shipping, and demonstrates how to build visual dashboards with Kibana to present the collected metrics, thereby enhancing the monitoring and control capabilities of cloud operations teams.

Docker's Datacenter offering enables enterprises to establish their own internal containers-as-a-service (CaaS) environments, integrating Docker into software delivery processes through a subscription model. The Docker Datacenter (DDC) includes components such as the Docker Universal Control Plane (UCP), Docker Trusted Registry (DTR), and the Commercially Supported Docker Engine, facilitating seamless application deployment from development to production. It offers cluster orchestration, cloud templates, and the ability to manage Dockerized applications both on-premise and in the cloud. UCP provides cluster management with role-based access control, supporting LDAP and Active Directory, while DTR addresses security concerns with similar authentication features. Deployment options for DDC include using AWS CloudFormation templates or a hosted CaaS solution operated by Docker. This setup allows for easy deployment and management of node clusters and services like NGINX across cloud or on-premise environments, emphasizing high availability and security features that enhance the efficiency of DevOps processes.

Logz.io, an AI-powered log analysis platform, has compiled its second annual list of tech, IT, and DevOps conferences for 2017 to help community members navigate the overwhelming number of events. The list includes notable conferences such as Dockercon, AWS Summits in various locations, and DevOpsDays in cities like Austin, Chicago, and Berlin, among others. While the exact details are still being confirmed, Logz.io is planning to sponsor or present at several of these events and encourages attendees to reach out if interested in meeting at one of the conferences. This list is meant to be a dynamic resource that will be updated throughout the year, and the company invites contributions from the community to ensure all relevant events are included.

AWS re:Invent 2016 showcased a diverse array of emerging technologies and startups that are contributing to the expansive ecosystem surrounding Amazon Web Services, underscoring its dominance in the cloud sector. Notable innovations included Fugue, which simplifies AWS deployment orchestration through its "Conductor" and Ludwig programming language, and CloudScreener, which offers a transparent pricing comparison tool for cloud infrastructure. Codefresh provides Docker-specific continuous integration and deployment solutions, while RiverMeadow facilitates seamless server migration across cloud platforms without downtime. FittedCloud distinguishes itself with machine learning-driven cost optimization for AWS services, ensuring efficient resource utilization. These technologies highlight the ongoing evolution and innovation within the cloud computing landscape, setting the stage for further advancements in future AWS events.

Kibana's Timelion, introduced by Elastic, enhances Kibana's capabilities in time series analysis and visualization, bridging some gaps with Grafana. Timelion allows users to combine data sources in a single visualization, perform advanced mathematical calculations, and visualize results using a straightforward syntax. With around fifty functions available—some experimental—users can calculate metrics like derivatives and moving averages. The tool requires Kibana 5.0 or later, and for older versions, a specific installation process is necessary. Timelion expressions begin with a data source function, and users can perform calculations such as memory usage analysis, visualize trends, and compare current data with past trends using offsets. Data formatting and visualization options are available, and Timelion can integrate external data sources via APIs. While still developing, Timelion provides powerful capabilities to complement Kibana's traditional visualizations, with the potential for integration into platforms like Logz.io. Users are encouraged to explore the built-in Timelion tutorial to overcome the learning curve associated with its use.

At AWS re:Invent, CEO Andy Jassy announced several new services and enhancements designed to strengthen Amazon's cloud offerings, touching on computation, databases, big data analysis, machine learning, and IoT. New instance types were introduced alongside Elastic GPUs for EC2, while Lightsail aims to simplify the creation of virtual private servers. AWS Aurora now fully supports PostgreSQL, making it more attractive to enterprises wary of Oracle and Microsoft pricing. The serverless Athena service allows direct querying of S3 with SQL, and new AI services, including Rekognition, Lex, and Polly, provide developers with tools for image analysis, conversational interfaces, and text-to-speech capabilities. Greengrass extends AWS functionality to IoT devices, enabling local execution of Lambda functions while maintaining cloud integration. These developments underscore AWS's commitment to remaining at the forefront of cloud technology innovation.