January 2021 Summaries

Mediatonic, a UK-based video game studio, successfully managed the launch of their popular game Fall Guys: Ultimate Knockout by utilizing advanced log management and data visualization techniques. To handle the large-scale multiplayer gaming environment, the technical operations team needed deep insights into the game’s mechanics, which they achieved by collaborating with Logz.io, a platform integrating Elasticsearch, Kibana, Prometheus, and Jaeger. This partnership enabled Mediatonic to centralize data, predict costs, and efficiently monitor server performance and user activity. By leveraging Logz.io's capabilities, they could visualize and manage gaming logs at scale, allowing rapid identification of server issues and optimization of gameplay experiences. This approach not only streamlined their operations but also provided unexpected insights and efficiencies, underscoring the platform’s utility in observability and monitoring for gaming and broader DevOps applications.

Sophos' Intercept X is a comprehensive endpoint security solution designed to protect against threats such as ransomware, zero-day exploits, and other vulnerabilities by employing a multi-layered approach that includes app lockdown, data loss prevention, and deep learning PUA blocking. This system is enhanced by Logz.io Cloud SIEM, which synchronizes data collected by Intercept X to streamline the correlation and prioritization of security events. Logz.io further enriches the security data and maintains specific rules to monitor various threats, such as suspicious runtime attempts and malicious URLs, while also providing configurable dashboards for comprehensive analysis and monitoring. The integration process involves setting up Filebeat 7 and configuring the Sophos API to enable seamless shipping of logs for further analysis, which is supported by prefabricated dashboards in Logz.io for detailed insights into malware and suspicious web activities.

Tomer Levy announced the decision to fork Kibana and Elasticsearch, a significant move that has received overwhelming support from various organizations, including large enterprises and small IT shops, as well as non-profits with humanitarian missions. The enthusiasm from companies and individuals willing to contribute has exceeded expectations. Following Elastic’s announcement, Logz.io has been collaborating with AWS to create a unified, open-source distribution that is community-driven and reliable. This initiative aims to bring together contributors from diverse sectors to participate in the development of this open-source project, with further plans to work closely with AWS and other partners in the coming weeks.

DevSecOps, the integration of development, operations, and security, remains a challenging yet critical methodology as organizations navigate the complexities of cloud-native applications spread across diverse environments. Security is often compromised during outages due to process shortcuts, increasing attack surfaces that require swift identification and remediation by security teams. The industry's shift towards embedding security within DevOps pipelines and automated systems aims to enhance security by default while demanding a blend of specialists and generalists to manage security tools and responsibilities. Despite the rarity of DevSecOps roles, a cultural shift is occurring where security becomes a shared responsibility, supported by emerging technologies like secure access service edge (SASE) and Security Orchestration Automation and Response (SOAR) tools. This transformation is part of a broader trend towards dynamic team structures, as seen in advanced organizations like Google, which utilize specialized Site Reliability Engineers to integrate security into operations. As security increasingly fades into the background through automation and embedded technologies, the promise of DevSecOps adoption continues to grow, driven by the need for high availability and secure systems in a rapidly evolving digital landscape.

Elastic's decision to change the licensing of Elasticsearch and Kibana from the open-source Apache-2.0 to a proprietary dual license under SSPL has sparked significant consternation within the open-source community, particularly among organizations dependent on these projects. In response, Logz.io, a company with substantial expertise in Kibana and Elasticsearch, plans to launch a new open-source distribution for these tools, aiming to keep them community-driven and under the Apache-2 license. This initiative seeks to foster collaboration, innovation, and service to the broader community, countering Elastic's move, which is perceived as a strategy to block competition, particularly from AWS, and monetize the ecosystem. Logz.io emphasizes its commitment to providing a superior observability platform, supporting the open-source ethos, and maintaining customer trust by advocating for open-source software's future.

Logz.io, a tech company with a strong presence in Boston and Tel Aviv, has been recognized by Built In Boston as one of the best places to work in the city's tech market, ranking high in various categories such as Boston’s 100 Best Places to Work and Boston’s 100 Best Midsize Companies to Work. This accolade reflects Logz.io's commitment to fostering a culture of innovation and providing exceptional perks and benefits for its employees, their families, and dependents, even amidst the challenges of adapting to a remote-first work environment due to the global pandemic. Built In Boston, a prominent online community for startups, has highlighted Logz.io's efforts in maintaining an enthusiastic and supportive workplace, which aligns with its goal of setting a standard for Boston startups. The recognition is particularly meaningful as it underscores the company's dedication to its workforce during a transformative period in the global work landscape.

AWS re:Invent 2020, held online for the first time over three weeks, featured a range of virtual activities and announcements, with Logz.io participating as a sponsor. The event attracted over 565 visitors to Logz.io's virtual booth, where they showcased their Cloud Observability Platform and introduced new services such as Distributed Tracing based on Jaeger and Prometheus-as-a-Service. The company highlighted their offerings in a variety of sessions, including a talk by CTO Jonah Kowall on DevOps Pulse 2020 results and a discussion by CEO Tomer Levy on AWS's role in supporting mid-to-late-stage ventures. Logz.io also engaged participants in a gamified Jam Session to explore operational and security issues using their tools, receiving positive feedback for the interactive format. Despite the virtual setting, interactive elements like raffles and giveaways contributed to maintaining the event's traditional fun and community engagement.

Logz.io now enables the integration of ModSecurity logs into its Cloud SIEM platform to enhance security monitoring by automatically parsing and displaying high-priority attack data through dedicated dashboards. ModSecurity, a widely used web application firewall, operates on the OWASP ModSecurity Core Rules Set and supports web servers like Apache HTTP, IIS, and NGINX. The integration involves setting up a shipping method from ModSecurity to Logz.io, requiring components like Apache2, the ModSecurity module, Filebeat 7, and the Logz.io public certificate. The platform provides a detailed view of security incidents through pre-configured dashboards and rules that highlight various attack vectors such as SQL injections, cross-site scripting, and remote command executions. This setup allows for a comprehensive analysis of security threats by organizing logs into visual formats like donut charts, making it easier to identify and investigate potential breaches. Logz.io's AI-powered observability offers a streamlined approach to managing security data across a network, providing users with a trial period to explore these capabilities.

Modern systems, characterized by cloud-native and microservices architectures, present significant monitoring challenges due to their complexity and the high volume of telemetry data they generate. Traditional monitoring tools like Graphite and StatsD have fallen short in addressing these needs, leading to the rise of Prometheus, which offers flexible querying and efficient metric scraping. However, Prometheus's scalability is limited by its single-node architecture, which struggles with long-term data retention and high cardinality metrics. To overcome this, modern time series databases (TSDBs) have emerged as a solution for long-term storage, allowing organizations to leverage Prometheus's strengths in metrics scraping and querying while providing scalable storage solutions for large volumes of telemetry data. The integration of TSDBs with Prometheus enables the retention of granular historical data without aggressive downsampling, making it an effective approach for managing the complexities of modern system monitoring.