November 2022 Summaries

Logz.io's product strategy in 2022 focused on addressing common challenges in observability, such as complexity, cost, and slow performance, by introducing several key initiatives to enhance their platform. These initiatives included reducing mean time to remediation, simplifying onboarding, and improving cost efficiency by filtering out unnecessary data. Notable updates included the launch of Logz.io Open 360, which integrates open-source technologies for unified log, metric, and trace analytics, and the introduction of features like Kubernetes 360 for health monitoring, Telemetry Collector for seamless data collection, Data Optimization Hub for cost control, and LogMetrics for trend analysis. The company also improved their Restore feature for faster log retrieval and introduced a Trace Sampling Wizard to simplify OpenTelemetry configuration. Looking forward to 2023, Logz.io aims to continue making observability more accessible, efficient, and cost-effective, while offering new users a 14-day free trial to explore these capabilities.

Logz.io has launched Open 360™, a comprehensive observability platform designed to provide modern engineering teams with end-to-end visibility across logs, metrics, and traces through an intuitive interface. Built on popular open-source tools like OpenSearch, Prometheus, Jaeger, and OpenTelemetry, the platform aims to streamline observability practices by reducing data ingestion costs and enhancing security. Open 360™ also introduces Kubernetes 360 for monitoring Kubernetes clusters and includes features such as a Telemetry Collector and LogMetrics for efficient data handling. This platform addresses the growing need for centralized visibility in distributed systems, helping organizations manage complexity and protect sensitive data while maintaining the flexibility and innovation of open-source solutions. Logz.io's initiative reflects its mission to offer an easy-to-use, scalable, and efficient observability solution, catering to over 1,400 customers worldwide.

In a discussion on the OpenObservability Talks podcast, Dotan Horovits and David Ostrovsky explore the transformative role of observability at Meta, particularly how it shifts from infrastructure-focused tasks to business-driven objectives aimed at enhancing both external and internal user experiences. Ostrovsky, a software engineer at Meta, contrasts his experiences at Meta with those at his previous company, Proofpoint, noting that while observability at Proofpoint was primarily engineering-driven, at Meta it starts with business discussions around service level agreements (SLAs) and focuses on the decision-making layer in fraud detection. Meta’s approach allows teams to leverage a robust infrastructure managed by separate dedicated teams, enabling Ostrovsky’s team to concentrate on higher-order observability that directly impacts product performance and user experience. This approach underscores the importance of addressing business-facing decisions through intelligent data analytics over telemetry data, which is key to enhancing developer observability and accelerating development cycles.

OpenSearch, an open-source alternative to ElasticSearch and Kibana, has achieved significant milestones since its inception 1.5 years ago, including garnering over 5,800 stars on GitHub and ranking among the top five search engines according to DB-Engines. The community-driven project, showcased at OpenSearchCon 2022 in Seattle, highlighted advancements in areas such as machine learning for security anomaly detection, improvements in plugin architecture, and developments in the OpenSearch Kubernetes operator. The conference featured discussions led by contributors like Lennart Koopmann and Nicholas Knize, along with roadmap insights from Amazon's Charlotte Henkle. Attendees learned about efforts to enhance OpenSearch's performance and scalability, with contributions from various teams, including Aiven and Opster. The event underscored OpenSearch's vision for unified observability and its potential transition into a Cloud Native Computing Foundation project, while encouraging further community involvement through forums, meetups, and public roadmaps.

In an increasingly complex digital landscape, engineering teams are struggling to effectively manage observability, despite having access to numerous tools and best practices. The 2022 DevOps Pulse Survey reveals a concerning trend: the Mean Time to Recovery (MTTR) has worsened, with a significant increase in respondents reporting MTTR over an hour from 2020 to 2021. This is compounded by the prevalent use of multiple observability tools, leading to an overload of noisy, valueless data, which many organizations lack the maturity or expertise to analyze effectively. The high costs associated with managing and storing excessive data further strain resources, while the intricate nature of modern environments adds to the challenge. The reliance on open-source tools, while popular, often results in incomplete solutions that fail to offer comprehensive observability, prompting some organizations to turn to proprietary solutions. Additionally, a deepening talent gap exacerbates these issues, as skilled personnel capable of navigating complex observability needs are scarce. Logz.io aims to address these challenges by leveraging customer feedback to develop solutions that unify capabilities, reduce complexity, and harness the flexibility of open-source tools, promising forthcoming innovations to improve observability environments.

Managed Security Service Providers (MSSPs) are rapidly evolving to meet the challenges posed by the increasing volume and complexity of threats, with a focus on improving margins and onboarding processes through high-quality, scalable tools. To achieve this, they are leveraging cloud-native services and automation, essential for efficient Security Operations Centers (SOCs) that can swiftly triage and respond to threats. The integration of Logz.io Cloud SIEM and Tines' SOAR solution demonstrates how these tools can work together to enhance data integration, multitenancy, and automated workflows, allowing MSSPs to operate efficiently at scale. Such integration facilitates seamless alert collection and case management, ensuring that both junior and senior engineers can handle threats effectively, supported by automated processes for incident identification and resolution. The shift towards best-in-breed, no-code solutions, as opposed to bundled SOAR options, highlights the need for robust, flexible automation, enabling security teams to access data from various tools and automate time-consuming workflows, thereby enhancing their ability to respond to threats quickly and at scale.