May 2021 Summaries

Logz.io has enhanced its Kubernetes monitoring capabilities by integrating OpenTelemetry instrumentation into its Helm charts, allowing users to extract traces and metrics from various data points in Kubernetes, such as Cadvisor, kube-state-metrics, Prometheus Node Exporter, and Prometheus Push Gateway. This new Helm chart, a fork of the main opentelemetry-collector chart, provides a comprehensive solution for Otel users by combining metrics and tracing, and it offers flexibility by allowing dependencies to be disabled if needed. Users can configure the chart by setting parameters such as the Logz.io shipping token, listener host address, and environment metrics name, with additional customization options available through the values.yaml file. For a detailed installation process, users are directed to consult the official documentation, and they are encouraged to subscribe to the blog for updates and access a free trial of the platform.

Dotan Horovits discusses the challenges of continuous deployment in modern systems, highlighting the frequent outages caused by infrastructure and code changes. He emphasizes the importance of correlating CI/CD events with system telemetry to identify problematic deployments quickly and reduce mean time to resolution (MTTR). Logz.io's Observability Platform has introduced a new Deployments feature in beta, allowing users to correlate deployment events with exceptions and errors flagged within Kibana Discover. This feature involves adding a simple RESTful API call to a CI/CD pipeline, enabling users to attach detailed information about deployments for efficient root-cause analysis. The deployment markers are now available in both the Log Management and Infrastructure Monitoring products, and users are encouraged to try the beta release and provide feedback to improve its usability.

AWS App Runner, an innovative service by AWS, enhances automation and capabilities for deploying, running, and scaling containerized workloads, building on the foundation of AWS Fargate without the need for manual maintenance and patching. Logz.io has integrated support for App Runner, aligning with its philosophy of providing flexibility to engineers by automating log and metric collection through services like Amazon CloudWatch and using tools like OpenTelemetry and Docker for sophisticated analysis and dashboarding. This integration aims to streamline production processes and help application developers focus more on deployment and application building rather than infrastructure management. The service facilitates continuous deployment and meets engineers at their code through GitHub integration, providing a flexible build configuration via both config-as-code and GUI options. The collaboration between AWS and Logz.io, supported by various AWS team members and led by Doron Bargo at Logz.io, underlines the potential benefits for developers seeking to solve business needs with less focus on infrastructure.

Open source projects are increasingly central to the tech industry, yet a trend has emerged where companies relicense their projects, such as Elasticsearch and Grafana, to more restrictive licenses, often creating tension between commercial objectives and community ideals. This shift is partly driven by companies’ struggles to align open-source efforts with profitable business models, leading them to protect their assets when faced with competition from other businesses. While some argue this is a sound business strategy, it can alienate users and prompt forks, as seen with AWS's OpenSearch. Successful examples of balancing open source with business interests include Red Hat and Canonical, which leverage service and product models while maintaining community engagement. Effective open source projects often rely on community-backed governance to ensure their growth and neutrality, as demonstrated by Kubernetes under the Cloud Native Computing Foundation. Ultimately, the text suggests that genuine commitment to open source involves relinquishing some control to foster a thriving, independent community.

Logz.io has developed a Cloud-Based Security Information and Event Management (SIEM) system designed to rapidly deploy and manage broad datasets cost-effectively while handling security events seamlessly. However, recognizing a need for more comprehensive security measures, Logz.io integrated a Security Orchestration, Automation, and Response (SOAR) solution in partnership with Siemplify. This integration aims to provide an end-to-end security solution by linking detection and response through a seamless workflow that includes ingestion, enrichment, detection, investigation, and response. The system uses pre-defined playbooks to automate responses to security incidents, allowing Security Operations Center (SOC) engineers and analysts to automate mundane tasks, reducing manual workload, minimizing incident response costs, and preventing alert fatigue. The platform supports a range of activities from continuous querying and data enrichment to executing remediation actions, with the flexibility to incorporate human intervention when necessary. This collaboration ensures efficient management of security incidents, promising to save time and resources while enhancing the security infrastructure.

Jaeger and OpenTelemetry, both part of the Cloud Native Computing Foundation, are prominent open-source tools for distributed tracing and telemetry data standardization, widely used to monitor microservices' performance. While Jaeger enables engineers to analyze individual traces for understanding latency issues, it lacks comprehensive service-level performance metrics, which the introduction of Aggregated Trace Metrics (ATMs) aims to address. By enhancing OpenTelemetry with a Span Metrics Processor, ATMs aggregate request, error, and duration (R.E.D.) metrics from trace data, allowing engineers to visualize these metrics with OpenTelemetry-compatible backends like Prometheus and Grafana. This open-source advancement is set to transform Jaeger into a more robust application performance monitoring tool by integrating ATMs directly into its UI, thus enabling a more holistic analysis of system behavior. The development encourages community engagement and contributions to further enhance observability capabilities, reflecting the positive reception from users eager to see its full potential realized.