May 2024 Summaries

Dynatrace AutomationEngine and Workflows enable users to build complex workflows with features such as conditions and loops, allowing for data fetching from external systems, notifications, and ticket creation. To enhance integration with third-party systems, the Dynatrace App Toolkit facilitates the creation of custom workflow actions, thereby expanding AutomationEngine's capabilities. The process involves setting up a Dynatrace environment, ensuring access to the necessary tools like the most recent LTS version of Node.js, and using an IDE such as VS Code. Users are guided through creating an app that queries electricity prices from an API and constructing a custom workflow action with specific input parameters. The guide includes steps to implement and test this action within the Dynatrace environment, addressing potential issues such as blocked requests, and concludes with instructions on processing the data and deploying the app.

Dynatrace, originally a small team from Linz, Austria, has evolved into a global organization with over 4,200 employees, facing both challenges and opportunities in scaling its operations. To address these challenges, Dynatrace is prioritizing its people and culture through a strategic transformation focused on four pillars: inspiring employee engagement, fostering career growth and leadership, attracting and retaining talent, and promoting a diverse and inclusive environment. Initiatives include measuring employee engagement through surveys, developing a Global Career Framework for clear career paths, launching a Culture Code to align values and behaviors, and fostering global connections through remote interaction initiatives. This ongoing transformation aims to create an engaged, empowered, and innovative workplace where employees can thrive, reflecting Dynatrace's commitment to continuous learning and adaptation in a dynamic market.

Logs are essential for DevOps and SRE teams to understand their tech stack, but extracting value from log data can be challenging due to issues in monitoring, finding relevant logs, and understanding them alongside other data like traces, events, and metrics. Manual tagging for monitoring is error-prone, potentially leading to critical gaps during outages or incidents. Dynatrace addresses these challenges through its Log Management and Analytics, which uses OneAgent for automatic log discovery and contextual enrichment within a Kubernetes environment. Logs are stored in Dynatrace Grail™, providing contextual insight across various Dynatrace Apps, such as Infrastructure & Observability, Databases, Clouds, and Kubernetes, and enabling easy navigation and analysis of logs without writing queries. This approach reduces the risk of prolonged outages and decreases the mean time to repair by offering streamlined log analytics, insight into log patterns, and effortless navigation of log contexts, thereby enhancing the effectiveness of teams in managing their infrastructure and applications.

Ally Bank sought to enhance its digital banking experience by gaining real-time insight into customer experiences without incurring high costs from multiple monitoring solutions. Partnering with Dynatrace, Ally Financial adopted AI-powered observability and full-stack monitoring, which enabled them to consolidate tools, enhance operational efficiency, and save millions annually. This collaboration allowed Ally to automate workflows and enrich metrics with predictive analytics, improving their customer service and reducing costs. In recognition of this successful partnership, Ally awarded Dynatrace the Ally Velocity with Quality award for their contribution to achieving these outcomes. Both companies have celebrated the mutual benefits of their collaboration, with Dynatrace also honoring Ally as a Digital Breakout Performer for its effective use of their tools. The partnership is marked by a commitment to innovation and delivering superior digital banking experiences, with Ally's continuous efforts to enhance efficiency and monitor cloud infrastructure costs.

In March 2024, Dynatrace's AI-powered platform became generally available on Microsoft Azure, offering users deep integration and instant insights into Azure resources through the Azure Native Dynatrace Service (ANDS). This integration allows for the deployment of Dynatrace OneAgent directly from the Azure Portal, enabling seamless data flow into the Dynatrace platform without configuration. Azure customers can now access Dynatrace core innovations, including the Grail data lakehouse for unified data analytics and the AutomationEngine and AppEngine for workflow automation and custom app development. The platform enhances cross-team collaboration with new user experiences like Dashboards and Notebooks, and it supports comprehensive monitoring of Azure workloads, ensuring a streamlined and efficient observability process. Additionally, it facilitates multiple Azure subscriptions under a single Dynatrace environment, providing a centralized source for observability data and enabling advanced analytics with the Dynatrace Query Language (DQL). This integration aims to accelerate cloud transformation and improve governance and compliance by allowing extended data retention and insightful resource management across Azure environments.

The text discusses the challenges and solutions related to stream processing in data-intensive applications, emphasizing the need for real-time processing and robust fault tolerance mechanisms in systems like Kafka Streams, Apache Flink, and Spark Structured Streaming. These frameworks enable high-level abstraction through directed acyclic graphs for data modeling, helping software engineers build scalable applications. However, fault tolerance remains a critical concern, with Kafka Streams showing volatile recovery behavior compared to Flink and Spark, which are more resilient. Optimizing configurations in Kafka Streams, such as adjusting the rebalance interval and using warm-up replicas, can improve recovery times but require careful tuning due to the complexity of the configuration space. The document suggests that further research is needed to develop new abstractions for automatic configuration tuning in large-scale industry setups, potentially involving advanced techniques like chaos engineering and large language models.

Dynatrace's blog post discusses how users can define metric-based custom events to enhance the capabilities of Davis AI for specific use cases. By creating these custom events, users can automatically analyze abnormal situations within their IT infrastructure and determine relevant impacts and root causes. Metric-based events can be tailored to alert users about critical changes, such as a drop in revenue or unusual service activity, while info-level events provide contextual information without triggering alerts. The configuration process involves setting event titles, severities, and scopes, with a structured metric selection and a preview feature to simulate alert outcomes based on historical data. The improved workflow for custom event alerting in Dynatrace provides users with flexibility in deciding whether an event should raise a problem, trigger a Davis AI analysis, or simply log an alert, while also offering threshold baseline recommendations to minimize alert spam.

Observability has become crucial for managing cloud and IT services, and the Dynatrace platform offers significant untapped value by transforming data into actionable insights in real-time. This technology enables executives to drive change, enhance productivity, mitigate risks, and optimize costs while reducing carbon footprints. By providing instantaneous answers without requiring pre-prepared schemas, Dynatrace simplifies complex data processing, thus enhancing business analytics and customer experience. It leverages hypermodal AI for proactive risk management, including improved availability and security compliance, and supports the transition to self-healing operations. Furthermore, the platform helps control tool sprawl and cost, consolidating tools and processes into a unified system, which promotes efficiency and sustainability. Dynatrace aims to empower executives to build stronger, data-driven organizations by streamlining operations and fostering innovation.

A message queue is a middleware mechanism that facilitates communication between diverse software components by queuing messages for asynchronous processing, allowing systems to handle tasks independently and efficiently, which is crucial for maintaining performance in complex distributed environments. It supports two primary styles: point-to-point, where a message is sent from one producer to one consumer, and publish-and-subscribe, where messages are broadcast to multiple subscribers simultaneously. Benefits of message queuing include scalability, resilience, load balancing, and asynchronous communication, making it essential for microservices architectures by decoupling components and providing robustness against system failures. The Advanced Message Queuing Protocol (AMQP) standardizes message exchange across platforms, ensuring consistent performance. Despite its advantages, message queuing presents challenges in observability due to its non-sequential processing and the difficulty in diagnosing message delivery issues. Observability platforms like Dynatrace address these challenges by using AI to monitor and analyze message queues, improving application performance and reliability. Popular message queue software options include Apache Kafka, RabbitMQ, and IBM MQ, each offering unique features and compatibility with various messaging protocols.

Akamas is a technology company specializing in AI-driven optimization for Kubernetes applications, partnering with Dynatrace to enhance performance, cost-efficiency, and reliability. The Akamas platform, designed by performance engineering experts, integrates with Dynatrace to provide insights into infrastructure health, FinOps, and security, offering automatic lifecycle management and alerting. By analyzing Dynatrace observability data, Akamas identifies optimization opportunities, such as reducing costs, improving reliability, and enhancing performance. In a practical example, the Akamas app optimized a Kubernetes environment monitored by Dynatrace, uncovering reliability and performance issues and identifying potential monthly savings. The solution allows users to define optimization goals, set constraints to maintain service level objectives, and apply changes automatically or with manual input. The app exemplifies how custom solutions can be developed on the Dynatrace platform to meet specific business requirements, highlighting the synergy between Dynatrace and Akamas in optimizing Kubernetes environments.

Fluent Bit 3.0 is the latest version of the telemetry agent designed to enhance data processing and observability pipelines with new features such as HTTP/2 support, new processors for data selection, and SQL processing capabilities for optimized data routing and storage. It offers improved scalability, performance, and efficiency, making it a cost-effective solution for managing logs, traces, and metrics. The release introduces best practices, including understanding data objectives, efficiently using plugins like Expect and content_modifier, and applying modifications at the data source to enhance pipeline efficiency. Fluent Bit 3.0 also emphasizes the importance of memory management and resource usage to prevent instability, while encouraging the use of Hot Reload for seamless configuration changes and the Fluent Bit Metrics plugin for Prometheus data exposure. The release aims to make observability pipelines more robust and reliable by offering a streamlined approach to data management and processing.

Red Hat and Dynatrace have partnered to enhance operational efficiency through the integration of Red Hat Ansible Automation Platform, Dynatrace Davis AI, and the Dynatrace observability and security platform. This collaboration aims to automate workflows, minimize disruptions, and improve mean time to resolution (MTTR) by enabling closed-loop remediation processes. By leveraging advanced AI for root cause and predictive analysis, the integration addresses the challenges organizations face in using observability and security data for automation, with a particular focus on reducing the time-consuming nature of problem remediation. Two implementation scenarios, involving event-based and rulebook-based remediations, highlight the flexibility and precision of the system in handling production issues, such as faulty deployments or security vulnerabilities. This strategic partnership is designed to streamline DevSecOps and SRE collaboration, providing a comprehensive framework for proactive problem-solving and efficient incident management.

Integrating Amazon Data Firehose with the Dynatrace Grail analytics engine allows enterprises to manage the increasing complexity of cloud workloads by providing real-time log data for effective observability and issue resolution. As cloud environments grow, the volume, velocity, and variety of log data expand, necessitating a robust solution like Dynatrace's AI-powered platform to analyze logs and mitigate impacts on business operations. While Amazon CloudWatch is a common tool for log collection, Data Firehose serves as a critical intermediary for streaming high-frequency logs, which can then be directly ingested into Dynatrace for comprehensive monitoring and troubleshooting. This integration simplifies log management by eliminating intermediary steps, enabling seamless data streaming from AWS to Dynatrace's Grail data lakehouse, where logs are processed and stored, enhancing the platform's capability to detect and address issues with minimal disruption. The setup process is straightforward, involving generating an API key and configuring AWS settings, allowing teams to view and analyze logs in context with other AWS services such as Lambda functions. This integration is available in Dynatrace version 1.290, offering users a scalable solution for log management and a direct link between AWS and Dynatrace environments.

As organizations increasingly adopt generative AI and cloud-native technologies, security teams face mounting challenges in protecting expanding attack surfaces while maintaining application performance. Despite the benefits of generative AI, such as accelerated development and increased productivity, concerns around data leakage and intellectual property misuse persist, with only a minority of organizations having policies in place to govern AI use. At the RSA conference, experts are set to explore how AI and security are intertwined with observability, impacting application security, vulnerability management, and threat detection. The convergence of observability and security data offers a promising solution to address these challenges by providing DevSecOps teams with comprehensive visibility into application security issues, facilitating real-time responses. As compliance requirements evolve, organizations are increasingly reliant on automation to manage risks and maintain regulatory compliance in complex multicloud and hybrid cloud environments, with a focus on understanding and mitigating an expansive attack surface.