March 2021 Summaries

Metrics Explore is a feature within Logz.io designed for in-depth analysis of Prometheus metrics, offering functionalities akin to Kibana Discover, such as easy querying and metadata exploration. It allows users to navigate metrics efficiently through organized lists and dropdown selections, and includes features like query autocomplete to simplify metric identification. Additionally, it supports distributed tracing with Logz.io and Jaeger, providing a comprehensive view of metrics across various services. The platform emphasizes the importance of metadata labels for metric segmentation and offers optimization suggestions for metric displays. A notable feature is the split screen, which enables simultaneous viewing of metrics and logs or comparisons of different data sources, enhancing the user's ability to analyze and contrast performance metrics effectively. Logz.io continues to develop its Metrics UI to leverage the capabilities of Prometheus-as-a-Service, with updates available through their blog subscription.

Logz.io has focused on enhancing its observability service by transitioning from using solely Elasticsearch for metrics storage to exploring alternative databases compatible with Prometheus to address cost and performance issues. Initially reliant on Elasticsearch, Logz.io faced challenges with high storage costs and suboptimal performance due to Elasticsearch's unsuitability for time-series data. To better accommodate users already utilizing Prometheus and to streamline cloud migration, Logz.io evaluated four options: VictoriaMetrics, M3DB, Thanos, and Cortex, considering factors like scalability, cost, and integration with multi-region cloud setups. Ultimately, M3DB, an open-source project from Uber, was selected for its full compatibility with Prometheus, despite some limitations in its community diversity and robustness. Logz.io has committed to contributing to and scaling M3DB to support its Prometheus-as-a-Service offering, leveraging Kubernetes for deployment and management.

Dotan Horovits provides a comprehensive guide on utilizing Logz.io Infrastructure Monitoring for visualizing metrics data through Prometheus dashboards, focusing on creating and customizing dashboards to effectively interpret data. The guide highlights the ease of starting with pre-built Prometheus dashboards available on Logz.io, which can be duplicated or customized for specific needs, including monitoring AWS cloud services, databases, and container environments. It also explains how to import existing Grafana dashboards into Logz.io by exporting them to JSON format, and offers a Python tool for bulk imports. For creating custom dashboards, the guide recommends defining relevant variables using queries in the Logz.io interface, emphasizing the importance of filtering metrics to refine data analysis. It explores the flexibility of panel visualizations within the Logz.io metrics UI and how PromQL queries can be used to parameterize panels, enabling users to conduct detailed investigations and link metrics with logs for enhanced observability. The platform offers a free 14-day trial, inviting users to explore its capabilities without commitment.

Dotan Horovits's tutorial on Logz.io's Prometheus-as-a-Service metrics solution provides a comprehensive guide for setting up infrastructure monitoring using Prometheus and Logz.io. The tutorial emphasizes the importance of metrics in monitoring system health by quantifying data points related to application or infrastructure performance, such as CPU usage, which can signal potential issues requiring further investigation through log analysis. Users are guided through the process of activating a Metrics service account on Logz.io, configuring Prometheus to send metrics to Logz.io, and managing storage settings to optimize resource use. The tutorial also highlights the integration of multiple Prometheus instances for centralized monitoring and encourages users to explore visualization and analytics tools in subsequent tutorials, all while offering a free 14-day trial for hands-on experience.

Logz.io has launched Prometheus-as-a-service, designed to simplify and centralize metrics management on their scalable cloud platform, reducing the complexity and overhead associated with managing Prometheus deployments. Prometheus, widely adopted by DevOps teams for cloud monitoring, is recognized for its ease of integration with distributed environments but also presents challenges in managing large volumes of metrics over time. Logz.io's service allows teams to maintain their existing Prometheus configurations while offloading data storage and analysis, thus unifying metrics with logs and traces for comprehensive observability. The platform supports easy migration of Grafana dashboards, enabling users to continue leveraging familiar analytics tools while benefiting from the unified, scalable infrastructure. Through this service, Logz.io aims to streamline monitoring operations by consolidating observability tools, enhancing troubleshooting, and alleviating the management burden typically associated with maintaining multiple open-source systems.

The text explores various domain-specific languages (DSLs) used in data querying and system observability, emphasizing their importance in fields like logging, monitoring, and telemetry. It highlights several key tools and languages, such as the ELK Stack for logging, Prometheus and Grafana for metrics, and Jaeger, OpenTelemetry, and Zipkin for tracing. The discussion underscores the role of SQL as a foundational language, influencing many DSLs, while also detailing the unique characteristics of other query languages like Apache Lucene for Elasticsearch, PromQL for Prometheus, Kibana Query Language (KQL), and Kusto Query Language for Azure Data Explorer. These languages are designed to enhance data querying capabilities and system performance by offering specialized features tailored to specific tasks, often building upon general languages to streamline complex operations. The text suggests that gaining familiarity with these DSLs is beneficial for professionals in the observability and monitoring sectors.

In March 2021, Microsoft revealed a cyber campaign by the Chinese nation-state threat actor known as Hafnium, exploiting zero-day vulnerabilities in Microsoft Exchange to exfiltrate data from targets like infectious disease researchers and defense contractors. In response, Logz.io swiftly implemented detection rules in its Cloud SIEM to identify Hafnium's use of web shells, focusing on Indicators of Compromise rather than the specific Exchange vulnerabilities. The attack exposed the disparity in cybersecurity capabilities, with smaller organizations being more susceptible due to limited resources, while those using cloud-based services like Office 365 or Managed Security Service Providers (MSSPs) demonstrated enhanced resilience. The incident underscores the necessity for affected organizations to promptly patch their systems, a complex task that can be prolonged, especially for critical infrastructure. Meanwhile, continuous monitoring and detection of Hafnium's evolving tactics remain vital, and Logz.io has pledged to support this effort while offering trial access to their security solutions.

Logz.io, a company known for its commitment to open-source technology, has been actively contributing to the development of distributed tracing tools like Jaeger and OpenTelemetry. The company's employees, Albert Teoh and Yogev Metzuyanim, discuss the balance between corporate objectives and community contributions, emphasizing the benefits and pressures of being involved in open-source projects. They highlight the collaborative nature of the open-source community, where developers from various companies, including competitors, work together to advance shared goals. The involvement in these projects has led to personal growth and recognition within the community, as exemplified by Albert's promotion to an official maintainer of the Jaeger project. Despite initial apprehensions about public scrutiny, the experience has been overwhelmingly positive, reinforcing Logz.io's dedication to being an integral part of the open-source ecosystem while enhancing its own technological capabilities.