September 2016 Summaries

Logz.io, a log analytics SaaS company, chose to build its platform on Amazon Web Services (AWS) due to its ability to support rapid growth, scalability, and high availability, alongside stringent security and compliance needs. Leveraging AWS's infrastructure, Logz.io can efficiently manage large-scale data ingestion and processing, ensuring fault tolerance and seamless service delivery even during data bursts. AWS's compliance with standards like SOC 2 and HIPAA, combined with its robust security features, helps Logz.io maintain high data security and protection. The company also benefits from AWS's advanced management and monitoring tools, such as CloudWatch and CloudTrail, which allow engineers to focus on enhancing business logic rather than managing infrastructure. Additionally, AWS's extensive ecosystem and community support reduce the learning curve and integration costs with other technologies, while its partner program aids in marketing and strategic positioning. Despite acknowledging the progress of competitors like Microsoft Azure and Google Cloud Platform, Logz.io found AWS's offerings more aligned with its needs, particularly for data-intensive and high-growth environments.

Docker Swarm is a native clustering tool designed to turn a pool of Docker containers into a single virtual host, facilitating scalable and transparent management across multiple hosts. Monitoring the health of such a distributed system is vital yet challenging, prompting the need for a centralized logging architecture. This guide outlines steps to create a Swarm cluster using Docker Toolbox, Docker Machine, and VirtualBox, and to implement logging by collecting event data, such as container status and actions, and shipping it to the Logz.io ELK Stack for analysis. The process involves setting up a local cluster with a Swarm manager and two nodes, using discovery tokens to form a cohesive cluster, and employing UNIX sockets and the Docker API to retrieve logs. These logs can be transferred into ELK via AWS S3 buckets or Logstash, with visualizations created in Kibana to monitor the cluster effectively. This approach ensures real-time visibility into the cluster's operations, enabling timely troubleshooting and ensuring optimal performance.

The blog post discusses effective strategies for monitoring Elasticsearch clusters, particularly after the release of Elasticsearch 2.0 and Marvel 2.0, which limited the availability of certain metrics. The authors developed a solution using a Docker image, "es-health," to initially monitor their clusters, but later integrated Graphite to capture real-time metrics for more complex analysis. By sending numeric metrics to Graphite via the "pickle" protocol and utilizing Grafana for visualization, they created a robust monitoring dashboard. The post emphasizes understanding normal metric behavior, identifying anomalies, and recognizing that different use cases for Elasticsearch may require unique monitoring approaches. It advises readers to develop personalized methods for monitoring rather than solely relying on generic online advice, underscoring the importance of visibility and adaptability in effective monitoring practices.

Gilly Barr discusses the challenges and solutions for running UI tests at Logz.io using the Intern framework and Selenium, facilitated by a Jenkins build server and Docker machines to perform UI tests after each commit. This process, while essential for deployment reliability, is fraught with "flaky tests" and bugs that arise in the Jenkins environment due to timing issues, which are not present when tests are run locally. To address these, Barr developed a method to wrap each web page object with a proxy called MethodLogger, which logs each method call to better pinpoint where a test fails, thus avoiding the time-consuming process of debugging with numerous console.log() statements. This innovation streamlines the testing process by providing clearer insights into the specific points of failure within the UI tests.