May 2016 Summaries

Logz.io, founded by security veterans from Check Point Software, offers an advanced log management and analysis solution to secure customer data around the clock. By integrating cutting-edge technology with robust organizational processes and a focus on security, the company employs a microservices architecture built on high coding standards and rigorous testing. It ensures data protection through AWS's secure infrastructure, Docker containerization, and network segmentation, alongside continuous monitoring for anomalies. Customer data is meticulously managed with encryption and dedicated data stores, ensuring full segregation and secure transmission. Logz.io also emphasizes access management and enterprise-grade security protocols, supported by a dedicated security operations center and a team of trained security architects.

Daniel Berman's blog post offers a curated list of ten essential resources for newcomers and more experienced users of the ELK Stack, a powerful tool for centralized logging and data analysis. The article highlights key platforms such as Elastic's website, EagerElk, and Tim Roes’ Blog for foundational and advanced learning. It also suggests using community-driven sites like StackOverflow, Reddit, and GitHub for troubleshooting and staying updated with ELK developments. Additionally, the post recommends DigitalOcean and DZone for detailed tutorials and ELK-related articles and encourages readers to explore Logz.io’s in-depth guides for mastering the ELK Stack. The author emphasizes the abundance of available resources and invites readers to contribute additional suggestions for enhancing the learning experience.

The text outlines a comprehensive guide on implementing a Payment Card Industry Data Security Standard (PCI-DSS) compliance dashboard using the ELK Stack (Elasticsearch, Logstash, Kibana) and OSSEC Wazuh, an open-source intrusion detection system. It highlights the importance of log management in meeting PCI-DSS requirements by tracking and monitoring access to network resources and cardholder data, which is essential for preventing and detecting data breaches. The guide details the installation of OSSEC Wazuh on Ubuntu servers, the integration with the ELK Stack, and the use of Amazon S3 for log syncing and shipping to Logz.io, a cloud-based log management platform. It also describes the creation of a PCI Compliance Dashboard in Kibana to visualize compliance data and trends, providing pre-made visualizations and dashboards for specific data types. Additionally, the text discusses alternative methods for integrating Wazuh with the ELK Stack using Docker and Logstash, emphasizing the role of these tools in enhancing security through monitoring, intrusion detection, and alerting capabilities.

ELK Apps, launched by Logz.io, is a free online library offering pre-made Kibana searches, visualizations, alerts, and dashboards tailored for specific environments and log types, allowing users to bypass the complex task of manually creating these visual tools. This feature has rapidly gained popularity among the Logz.io community due to its ease of use, enabling users to install and customize apps with a single click while also offering the option to contribute their own apps to the library. Contributors can share a range of Kibana resources, including saved searches and dashboards, by submitting them for review and potential inclusion in the library, which currently hosts 142 apps covering various logging environments such as Apache, NGINX, and AWS. Users can create a free Logz.io account to participate and contribute, adding value to this collaborative platform that supports a wide array of logging scenarios.

Logz.io conducted an internal benchmarking study to determine the most effective way to utilize Elasticsearch clusters on AWS, focusing on indexing performance. They developed a custom stress test tool to simulate their specific logging use case and tested various instance types, including those with ephemeral storage and EBS (Elastic Block Store). Despite Elastic's advice against using EBS, the SSD-backed EBS performed exceptionally well, especially when instances were EBS-optimized. The study highlighted the importance of calibrating bulk sizes and revealed that while the c4.2xlarge instance was more cost-effective, the m4.2xlarge offered more RAM, which is crucial for Elasticsearch performance. Ultimately, Logz.io concluded that the m4.2xlarge was the best fit for their needs, emphasizing that the choice of instance type depends on specific use cases and infrastructure requirements.