July 2018 Summaries

The DevOps Pulse 2018 study reveals significant insights into the current state of DevOps practices, particularly focusing on security challenges and the integration of new technologies. Despite 54% of DevOps teams being tasked with security operations, a substantial 71% lack familiarity with DevSecOps best practices, highlighting a critical gap in knowledge and preparedness. The survey also indicates that the tech industry is underprepared for GDPR, with 39% of respondents admitting they are not ready, exposing companies to potential liabilities. Furthermore, while many believe their organizations are diverse, the overwhelming male representation among respondents suggests otherwise. The study notes a significant shift towards Kubernetes adoption, with over 60% using container orchestration tools, and the rise of serverless technology, now used by 42% of IT professionals. Additionally, continuous integration and deployment strategies are increasingly crucial, with 91% and 85%, respectively, either implementing or considering these practices.

Prometheus and Graphite are both open-source tools for monitoring time series data, but they differ significantly in their capabilities and applications. Prometheus, developed by SoundCloud in 2012, is a comprehensive monitoring system that actively collects, stores, queries, and alerts based on time series data, making it suitable for dynamic, service-oriented monitoring with features like a flexible query language and built-in support for alarms. It also offers a wider range of client libraries and scalability options, such as partitioning and replication. In contrast, Graphite, introduced by Orbitz in 2006, is a simpler tool focused on passively storing and graphing numeric time series data, relying on external tools for data collection and alarm generation. This simplicity and its compatibility with existing collection tools like fluentd and collectd make Graphite ideal for long-term data storage and systems with established infrastructures. While Prometheus requires a steeper learning curve due to its integrated features, it offers a more robust solution for new implementations requiring comprehensive monitoring capabilities. Both tools have active open-source communities and are widely used across various industries for their reliability and real-time monitoring capabilities.

Request tracing is an essential tool for gaining insights into distributed systems by tracking operations across various components, allowing engineers to identify bottlenecks and latency issues. It complements logs and metrics, forming a comprehensive telemetry solution that aids in debugging and resolving production problems. Zipkin and Jaeger are two popular request-tracing tools, each with its unique strengths. Zipkin, inspired by Dapper and developed by Twitter, is now maintained by a dedicated community, while Jaeger, built by Uber, is a Cloud Native Computing Foundation project. Both offer similar architectures with a focus on capturing and analyzing trace data, but Jaeger holds an edge in library instrumentation by leveraging Open Tracing libraries. Deployment considerations are crucial; Jaeger integrates seamlessly with Kubernetes environments due to its CNCF affiliation, whereas Zipkin's simpler, single-process model may be better suited for non-containerized settings. The choice between the two depends on existing infrastructure, language support, and community engagement, with Jaeger's compatibility with Zipkin's API offering additional flexibility.

Elasticsearch 6.3 introduced significant updates, including SQL support that enhances accessibility for users familiar with relational databases, allowing them to execute SQL queries on data indexed in Elasticsearch. This new feature, part of the X-Pack under a special Elastic license, enables querying using SQL statements without the need for additional plugins. Users can run SQL queries through REST API, a console tool within Kibana, a SQL CLI tool, or even integrate it with Java applications via a JDBC driver. The SQL support, still in experimental mode, allows for a range of queries and results display formats, making Elasticsearch more user-friendly, despite some limitations like the inability to execute JOIN statements. Additionally, the Translate API can convert SQL statements into Elasticsearch Query DSL, providing further flexibility in data querying and manipulation.

Kubernetes, an increasingly popular open-source system for automating deployment, scaling, and management of containerized applications, presents distinct advantages and challenges when considering hosted versus self-hosted services. While self-hosting Kubernetes demands considerable expertise in network and Linux configurations and ongoing maintenance to ensure security, hosted services offer a more accessible solution by managing infrastructure and updates. The article analyzes five leading hosted Kubernetes services: Google Cloud Kubernetes Engine, Azure Kubernetes Service, Amazon Elastic Container Service for Kubernetes, IBM Cloud Container Service, and Rackspace Kubernetes as a Service, highlighting their features such as automatic updates, load balancing, auto-scaling, node pools, and multi-zone capabilities. Google Cloud is noted for its comprehensive feature set and quick updates, while IBM and Rackspace are recognized for offering bare metal machines, which could benefit CPU-bound processes. Ultimately, the choice of service depends on specific needs such as scalability, ease of management, and performance requirements, with considerations for avoiding vendor lock-in and optimizing resource utilization.

Modern companies cannot afford downtime due to traffic spikes, which is why AWS's Elastic Load Balancer (ELB) is crucial for maintaining elasticity and scalability in applications. ELB enables applications to distribute user requests across multiple resources, ensuring high availability and flexibility, and supports autoscaling through Auto Scaling Groups and monitoring via CloudWatch. AWS offers three types of load balancers: Classic Load Balancer for EC2-Classic networks, Application Load Balancer (ALB) for advanced HTTP/HTTPS traffic routing, and Network Load Balancer (NLB) for handling high loads with low latency and static IP addresses. Monitoring and logging via AWS tools like CloudWatch and VPC Flow Logs help track metrics such as HealthyHostCount and TCP resets, but the analysis of logs stored in AWS's S3 requires additional tools like Logz.io’s ELK Stack for deeper insights. Ultimately, while AWS load balancers effectively route traffic, the responsibility of analyzing and utilizing the data to enhance application performance lies with the user.