July 2024 Summaries

On July 19th, 2024, a routine software update from CrowdStrike caused widespread IT outages, affecting various industries and highlighting the importance of rapid recovery capabilities for business resilience. The incident underscored the value of observability and monitoring tools, such as those offered by Dynatrace, which provide real-time data, AI-driven analytics, and synthetic monitoring to enable swift identification and resolution of issues. Companies using Dynatrace's tools, including real user monitoring and Dynatrace Query Language (DQL), were able to efficiently manage the crisis by maintaining situational awareness, prioritizing critical systems for remediation, and ensuring service continuity. Real-world examples demonstrated how these tools helped organizations quickly recover, maintain operational stability, and protect customer experiences. The event emphasized the growing necessity for comprehensive observability solutions in an increasingly digital world to prepare for future challenges and ensure robust IT infrastructure.

As financial institutions work towards compliance with the Digital Operational Resilience Act (DORA), which mandates resilience against operational disruptions by January 2025, Dynatrace offers solutions to ease the compliance burden through automation and analytics. DORA sets forth five pillars—ICT Risk Management, ICT Incident Reporting, Digital Operational Resilience Testing, ICT Third-Party Risk Management, and Information and Intelligence Sharing—to ensure robust risk management and operational resilience. Dynatrace's platform utilizes observability and security data, supported by hypermodal Davis AI, to automate up to 80% of the repetitive tasks associated with DORA compliance, addressing key challenges such as digital ecosystem complexity, third-party risk management, resource constraints, and seamless integration with existing processes. The platform provides comprehensive solutions, including full-stack observability, runtime vulnerability analytics, automated compliance monitoring, and seamless integration, turning compliance challenges into opportunities for innovation while maintaining security and operational efficiency.

The rise of generative AI and cloud-native technologies is escalating cybersecurity concerns, as organizations strive to secure increasingly complex cloud applications against expanding attack surfaces. While generative AI offers the promise of faster development and increased productivity, it simultaneously presents new security challenges that DevSecOps teams must address. Current practices often leave a significant portion of security events unexplored, leading to delays in understanding breaches. To combat these issues, organizations are merging observability with security data, providing real-time visibility that enhances threat detection and response capabilities. AI is being leveraged to automate and accelerate DevSecOps processes, enabling proactive threat management and data privacy protection. As multicloud environments grow in complexity, they pose additional security challenges, necessitating a unified observability and security strategy to streamline threat detection and remediation. This approach empowers organizations to maintain a robust security posture by utilizing AI to prioritize threats and automate responses, ultimately enhancing cyber resilience and reducing the mean time to repair issues.

In today's interconnected digital environment, understanding third-party vendors' quality control and release processes is crucial for maintaining business resilience, as highlighted by the issues caused by a faulty CrowdStrike update. This blog outlines key considerations for assessing vendors, covering aspects such as development and QA processes, security practices, customer support, application monitoring, and system resiliency. It emphasizes the importance of verifying that vendors not only provide advanced technology but also demonstrate reliability, security, and effective customer support strategies. Organizations are urged to engage in proactive risk management by asking pertinent questions about vendors' testing processes, security risk management, support availability, monitoring strategies, and incident management plans. By thoroughly evaluating these areas, businesses can better prepare for potential outages and enhance their resilience against disruptions in their software supply chain.

Kubernetes, while an industry standard for container orchestration, presents significant complexity challenges as organizations scale their deployments, as evidenced by PicPay's experience. PicPay, a leading digital wallet and bank in Latin America, faced difficulties in managing their extensive Kubernetes infrastructure, which includes over a hundred clusters and thousands of microservices. The complexity led to fragmented IT stacks and inefficient log-heavy analysis methods, prompting PicPay to adopt a strategic Kubernetes platform engineering approach. This approach involved breaking down large clusters, standardizing infrastructure, and integrating metrics, traces, and logs to improve observability and cost efficiency. Partnering with Dynatrace, PicPay consolidated its toolset, enhancing visibility and scalability while reducing maintenance costs. This strategic shift underscores the importance of platform engineering in managing Kubernetes complexity, emphasizing the need for a unified toolset and early implementation to simplify infrastructure and improve operational efficiency.

In the wake of the CrowdStrike update outage, organizations are increasingly focusing on enhancing their software update and delivery practices to ensure business resilience and maintain user trust. The blog discusses the importance of adopting modern deployment techniques like progressive delivery strategies—such as rolling, blue/green, canary deployments, and feature flagging—to mitigate risks and minimize downtime during software rollouts. These methods provide a controlled and gradual approach to releasing software, reducing the impact of potential failures. Integrating tools like the Dynatrace platform into continuous delivery processes offers automated monitoring and validation, allowing teams to proactively identify issues and respond swiftly, thus safeguarding the customer experience. By embedding Dynatrace's observability into CI/CD pipelines, organizations can enhance their pre-release and post-deployment phases, ensuring high-quality and stable software releases. This integration ultimately helps prevent outages and fosters a more reliable and competitive software delivery process.

Zero Trust Architecture (ZTA) is a cybersecurity model based on the principle of "never trust, always verify," designed to enhance security for organizations by addressing threats from both inside and outside a network. The model's seven pillars guide securing environments, emphasizing the importance of a mature application and workload pillar, which includes inventory management, DevSecOps frameworks, software risk management, user access control, and continuous monitoring. The National Security Agency (NSA) has issued guidance for advancing the maturity of this pillar, highlighting robust identity management and automated monitoring for full transaction visibility. This is crucial for U.S. government agencies, especially after Executive Order 14028 mandated ZTA for civilian agencies to protect national security information. The Dynatrace platform supports this by providing comprehensive observability and security analytics, enabling agencies to proactively manage vulnerabilities and protect critical data. Through tools like Runtime Vulnerability Analytics, Dynatrace offers real-time insights and automatic attack detection, helping agencies adapt to evolving threats and maintain a strong security posture in their journey towards zero trust maturity.

The Dynatrace Troubleshooting Community is an online platform where users can access expert advice and solutions for integrating and monitoring technologies on the Dynatrace platform. This community offers a wealth of articles written by Dynatrace experts, providing quick answers to common issues, and now includes contributions from the company's Engineering, Customer Success, and Support teams. Users can engage directly with support through chat or support tickets, while also benefiting from 24/7 self-service options and a broad range of troubleshooting tips. The community fosters collaboration and knowledge sharing, with resources available for specific technical issues such as Kubernetes monitoring and SSO troubleshooting. Dynatrace encourages feedback to continually enhance the platform, aiming to provide the best possible self-service experience for its users.

OpenTelemetry is an open-source observability project designed to enhance the understanding of application behavior in production environments by providing a standardized approach to collecting, processing, and exporting telemetry data, including metrics, logs, and traces. It offers a unified set of platform-agnostic APIs, libraries, agents, and instrumentation standards that ensure interoperability and vendor neutrality, allowing organizations to integrate multiple observability tools without reinstrumenting applications. By standardizing data formats and collection methods, OpenTelemetry facilitates comprehensive observability, enabling developers to gain deeper insights into system performance, identify inefficiencies, and proactively address potential issues. The project is continuously evolving with contributions from a vibrant community, ensuring alignment with the latest industry standards and technological advancements, making it a future-proof solution for modern software development practices.

Dynatrace SaaS version 1.297 introduces several updates and enhancements, focusing on improving security collaboration, data ingestion, and dashboard customization. The release allows for enriched vulnerability findings with runtime context and supports case sharing in the Security Investigator app, enhancing collaborative security threat investigations. A notable change includes the removal of analyzer names from Davis® Analyzer response logs, now available in the URL via API calls. The update also supports the automatic mapping of vulnerability findings in the Open Cybersecurity Schema Framework (OCSF) 1.1.0 format to the Semantic Dictionary conventions. Dashboards have been enhanced to allow tile-specific timeframes, enabling more granular data presentation, and a new matcher feature in log monitoring highlights the origin of log sources. The release resolves numerous issues across components like Dynatrace Cluster, Synthetic Monitoring, and Session Replay, addressing problems such as incorrect event merging, connectivity issues, and billing anomalies. Additionally, the Dynatrace Hub now facilitates app installation requests, streamlining user-admin interactions.

Dynatrace has collaborated with Microsoft Azure App Service to enhance observability for Linux App Service by employing the Sidecar Pattern in containerized computing. This approach allows for seamless integration of monitoring without altering the main application container, as the sidecar container operates independently to handle tasks like metrics collection and log capture. The Sidecar Pattern enables effortless integration, improved scalability, and enhanced observability, allowing organizations to update monitoring tools without affecting application performance, thus ensuring zero downtime. As an Azure Native ISV Services partner, Dynatrace leverages this partnership to deliver innovative solutions that drive digital transformation and business value for customers, with the Sidecar Pattern currently available in public preview.

In July 2024, a defective update from cybersecurity company CrowdStrike caused widespread system crashes on Windows-based servers, leading to significant global service disruptions across various industries. The update resulted in a "blue screen of death" (BSOD) and forced affected servers into boot loops, making them unresponsive and requiring complex, manual remediation. Dynatrace, an observability platform, assists organizations in quickly identifying and prioritizing affected systems using its advanced monitoring capabilities, such as Smartscape technology, which maps dependencies and critical services. This enables companies to restore services efficiently by focusing on their most crucial business applications, minimizing downtime and resource expenditure. Despite the severity of the outage, Dynatrace’s own services remained unaffected, though the incident highlighted the importance of comprehensive monitoring to ensure business resilience.

In the complex and fast-paced digital environment, managing IT infrastructure can be likened to playing a game of Jenga, where even minor changes can destabilize the entire system. Dynatrace addresses this challenge by integrating Synthetic Monitoring with visual workflows, offering a robust solution to maintain stability and support growth through automation. This integration enhances DevOps processes by enabling automated release validation, validating digital infrastructure changes, and offering customizable synthetic scheduling, all of which ensure performance and functional criteria are met. The approach leverages observability to provide insights into IT infrastructure, improving incident response and resolution times. By incorporating Synthetic Monitoring into delivery pipelines and using it alongside tools like Site Reliability Guardian, organizations can proactively assess the impact of changes on user experience and maintain system reliability and efficiency. The flexibility provided by event-driven architecture in Dynatrace Workflows allows for customized scheduling of synthetic monitors to adapt to varying operational demands, offering a strategic advantage in managing digital infrastructure effectively.

Dynatrace has expanded its Synthetic Monitoring capabilities with the introduction of Network Availability Monitoring (NAM), featuring new synthetic test types like ICMP Ping, TCP port check, and DNS, which provide 24/7 insights into the health and availability of infrastructure and network layers. These enhancements aim to address the limitations of existing HTTP and browser monitors by offering deeper insights into the transport and network layers, crucial for diagnosing complex issues in modern IT environments. The NAM tests, which can be customized for packet size and success rates, are designed to be easily deployable and manageable, integrating seamlessly with existing Dynatrace infrastructure monitoring tools. They allow users to perform comprehensive monitoring across multiple hosts from a single test, ensuring that any changes in network configurations are automatically reflected in the test setups. Furthermore, Dynatrace Managed users gain access to NAM's powerful APIs and reporting capabilities to maintain oversight of network health, while the new Synthetic app in Dynatrace SaaS environments facilitates the configuration and reporting of these network availability monitors.

Syslog is a critical protocol for monitoring systems and network devices by sending event data logs to a central location, yet it poses challenges due to its lack of authentication and variable flexibility among producers, along with inconsistent support for Transport Layer Security (TLS). Dynatrace addresses these challenges by integrating syslog data into its AI-powered observability platform through the Environment ActiveGate edge component, enabling seamless and secure ingestion without the need for complex configurations or additional components. This integration allows organizations to make syslog data actionable within the context of their infrastructure and cloud services, enhancing automation, security, and resiliency while reducing manual work and maintenance efforts. By using the Davis® AI for automatic problem detection, businesses can improve their mean time to identify and repair issues, thus increasing overall operational efficiency. The integration process is streamlined, requiring only a simple configuration change, and supports secure message exchanges, optimizing traffic volume and maintenance in protected networks.

As organizations increasingly adopt cloud-native technologies, traditional IT monitoring methods are proving inadequate for meeting modern business demands, prompting a shift toward business observability. This approach enhances IT operations by integrating metrics, traces, and log data, offering real-time oversight essential for data-driven decision-making and risk mitigation. Business observability aligns IT activities with strategic objectives, improving operational efficiency, security, compliance, customer experience, and resource optimization. With the support of partners like Deloitte, businesses can effectively navigate digital transformation and maintain a competitive edge by leveraging insights to optimize processes, ensure security, and foster innovation. The transition to business observability ultimately allows organizations to better harness IT capabilities, improving customer satisfaction and positioning them as leaders in the digital landscape.

Dynatrace has enhanced its data management capabilities by introducing record-level hard deletion, a process that aligns with privacy laws and ensures secure compliance while maintaining data integrity. This feature is implemented in Dynatrace Grail™, a high-performance data lakehouse designed for automated data collection and real-time querying of extensive datasets. The deletion strategy is supported by retention-based and on-demand deletion policies, allowing for customizable retention periods and precise control over which data is permanently erased. By enabling hard deletion, Dynatrace eliminates the risks associated with soft deletion, such as accidental exposure or unauthorized access, thereby bolstering data security and customer trust. The Grail Storage Record Deletion API facilitates targeted data removal, ensuring that only irrelevant data is deleted while preserving valuable business data. Furthermore, the Privacy Rights interface streamlines the deletion process and offers a dashboard for tracking requests, ensuring transparency and compliance. These innovations reflect Dynatrace's commitment to providing robust and transparent data protection solutions.

Dynatrace offers a new SaaS Upgrade Assistant app designed to facilitate the transition from Dynatrace Managed to Dynatrace SaaS, enabling users to leverage Grail-based innovations such as AppEngine and Davis CoPilot. Grail, a purpose-built data lakehouse, is integral to converging real-time, historical, and predictive analytics on a unified platform, although its capabilities are exclusive to SaaS environments. The Upgrade Assistant automates the migration process by importing configuration settings from Managed environments, minimizing manual effort and disruptions while ensuring a smooth upgrade. The tool provides two editing modes for configuration updates and includes a preview mode to help track migration progress, supporting cluster administrators in managing complex environments. Available starting with Dynatrace Managed version 1.294, the app is accessible via Dynatrace Hub and aims to streamline upgrades with minimal downtime and configuration issues.

The new Dynatrace extension simplifies network management by automating the discovery and inventory of SNMP devices, allowing for more efficient monitoring and management across large networks. This extension integrates autodiscovery and monitoring into a single tool, reducing manual tracking, costs, and errors while enhancing operational excellence. SNMP devices are discovered and listed in the Dynatrace platform, where additional extensions can be installed for device-specific monitoring, and users are notified of anomalies or threshold breaches. The setup involves configuring IP subnets, ranges, or individual addresses, and requires SNMP credentials for discovery. Discovered devices are added with minimal properties, and the Dynatrace API can automate full observability enablement. Future updates will further integrate SNMP device management into Dynatrace Apps, enhancing user experience and functionality for both Dynatrace SaaS and Managed customers.

Dynatrace's SaaS release version 1.296 introduces significant updates and enhancements aimed at optimizing IT infrastructure and application monitoring. Key features include the use of AI for performance and error optimization through Opportunity Insights, which leverages real-user data to improve business outcomes. The integration of Synthetic Monitoring with visual workflows enables automated infrastructure stability and growth management. New Network Availability Monitoring tests have been added to extend synthetic monitoring to network layers, providing comprehensive, 24/7 insights into infrastructure health. The release also introduces changes in code-level vulnerability monitoring, allowing for resource attribute-based matching instead of using process group IDs, and enhances API query parameter validation. Additionally, there are improvements in log monitoring with case sensitivity adjustments, the retirement of Office 365 connectors within Microsoft Teams, and the ability to ingest third-party security events via OpenPipeline. The platform also offers new visualization tools, like honeycomb visualization for monitoring infrastructure metrics, and updates to notebook and dashboard management, including version control and query limit settings.

The Dynatrace Managed release version 1.296 introduces several enhancements, including expanded Synthetic Monitoring capabilities through Network Availability Monitoring (NAM) with new synthetic test types like ICMP Ping, TCP port check, and DNS, providing continuous insight into the health and availability of infrastructure and network services. This release also changes the update process to allow faster upgrades by skipping certain versions, requiring users to adapt their testing approaches accordingly. Key improvements include the upgrade of Cassandra nodes to version 4.0.13, streamlined alerting for Kubernetes, and enhanced AWS and Kubernetes monitoring features. Breaking changes involve stricter parameter validation in the Deployment API, and operating system support updates are outlined, with some systems being phased out by the end of 2025. The release also addresses multiple resolved issues across various components, such as Application Security and Cluster, improving functionalities like credential handling, monitoring configurations, and issue resolution in high-availability clusters.

The Dynatrace Amplify 2024 event, an annual sales kick-off for partners, celebrated outstanding achievements in innovation and collaboration across global regions through its Partner Awards. This event underscores the crucial role of strategic partnerships in driving digital transformation, offering insights into cloud infrastructure, application performance, and AI-driven solutions. Dynatrace honored partners like DXC Technology, which received the Global Cloud Partner of the Year award, for their exceptional contributions to intelligent automation and customer satisfaction. Regional winners, including Accenture in EMEA and AHEAD in North America, demonstrated remarkable agility and expertise in their respective markets. The awards highlight the vital importance of collaboration in advancing technological capabilities and ensuring seamless digital experiences for customers, while also empowering partners with resources to stay competitive in a rapidly evolving digital landscape.

Artificial Intelligence (AI) holds significant transformative potential across industries, yet a high failure rate in AI projects, as reported by Gartner, often stems from poor data quality and insufficient understanding of AI requirements. To address these challenges, robust data management and strategic planning, including cloud-based models and large language models (LLMs), are imperative. Data observability, which involves monitoring the quality, lineage, and performance of data systems, is critical for ensuring the success of AI projects. Tools like Dynatrace offer features such as data freshness, volume, distribution, schema monitoring, and lineage tracking to enhance data reliability. Beyond data, AI observability focuses on monitoring model performance, explainability, and drift detection to ensure models operate effectively in real-world applications. Combining data and AI observability provides a comprehensive view of both pipeline infrastructure and model performance, facilitating swift issue resolution and compliance with governance standards. Dynatrace provides organizations with the necessary tools for both data and AI observability, enabling them to develop reliable and transparent AI applications that align with industry standards and ethical guidelines.

Fluent Bit, a widely used log collection and processing tool for Kubernetes, has integrated with Dynatrace to enhance the ease of onboarding and improve log management and analytics. This integration provides users with a ready-to-use Fluent Bit configuration to stream data to Dynatrace, thereby simplifying the troubleshooting process by allowing platform and development teams to examine Kubernetes logs in context, enriched with tracing metadata. By facilitating a clearer understanding of Kubernetes platforms and application behaviors, the integration aids in rapid troubleshooting and problem resolution within complex cloud-native environments. Users can quickly deploy the Fluent Bit installation via a DaemonSet or integrate it into existing setups, ensuring logs are enriched for context-rich analytics. The Dynatrace integration not only simplifies troubleshooting but also enhances the use of logs for health checks, security incident investigations, and understanding issues detected by AI, offering a comprehensive approach to log management and analytics in Kubernetes environments.

CVE-2024-6387, dubbed the regreSSHion vulnerability, is a critical Remote Code Execution (RCE) flaw found in the OpenSSH server on glibc-based Linux systems, identified by the Qualys Threat Research Unit. This vulnerability arises from a signal handler race condition in the OpenSSH server, potentially allowing unauthenticated remote attackers to execute code as root, posing a significant security threat. Despite its severity, exploiting this vulnerability is challenging due to its race-condition nature, requiring numerous attempts for successful exploitation, and no effective exploits have been reported thus far. The vulnerability impacts certain versions of OpenSSH, excluding OpenBSD systems, which preemptively implemented a secure mechanism to prevent such issues. Mitigation strategies include patch management, access control, and network segmentation, while detection can be achieved by analyzing sshd logs for suspicious activities using Dynatrace's DQL queries. Although the vulnerability is difficult to exploit, organizations are advised to assess and mitigate potential risks to secure their systems adequately.

A significant communication gap between security leaders and top executives is exposing organizations to heightened cyber threats, particularly as AI-powered attacks become more prevalent. The 2024 CISO report by Dynatrace highlights the critical importance of bridging this gap by focusing on system visibility and security, aiming to improve collaboration and reduce vulnerabilities. Despite the increased involvement of executives in cybersecurity discussions, there remains a disconnect due to differing priorities and a lack of understanding of technical risks, such as application security weaknesses. The report emphasizes the need for CISOs to translate technical threats into business risks that executives can comprehend, advocating for improved communication strategies and the adoption of DevSecOps automation to enhance security measures. Furthermore, the report underscores the necessity for modernized security practices in response to the complex, AI-driven threat landscape, urging organizations to integrate AI and automation to gain comprehensive visibility and protection. Dynatrace's platform is presented as a solution that integrates observability and security data, aiming to eliminate blind spots and assure the security of cloud-native applications.

AWS Application Migration Service, in partnership with Dynatrace, enables organizations to migrate on-premises workloads to the AWS cloud with enhanced efficiency, reduced downtime, and improved performance monitoring. This integration allows Dynatrace's AI-powered observability to automatically detect performance issues and security concerns during and after the migration process, ensuring seamless, efficient, and secure transitions. AWS Application Migration Service uses an agent-based approach for rapid and reliable migrations, maintaining operational continuity for applications. AWS Systems Manager Distributor facilitates the secure distribution and installation of software packages like Dynatrace OneAgent across both AWS and on-premises environments. This collaboration between AWS and Dynatrace provides organizations with tools to accelerate their migration efforts, eliminate uncertainty in cloud architecture design, and maximize the benefits of AWS cloud adoption.