August 2024 Summaries

Organizations often adopt Microsoft's Azure Functions as part of their cloud modernization strategy due to its serverless solution that enables event-driven code deployment without worrying about infrastructure management. However, there are certain security and performance optimization challenges to consider when working with Azure Functions, such as DDoS attacks susceptibility and ensuring optimal function performance. To optimize Azure Function performance, it is crucial to choose an appropriate hosting plan based on the application's requirements and expected traffic levels. The four major function app hosting plans offered by Azure are: Consumption plan, Premium plan, Dedicated plan, and Container Apps plan. Each plan has its own benefits and trade-offs in terms of performance, scaling behavior, resource allocation, and cost implications. Monitoring how the chosen hosting plan impacts cloud spend is essential to ensure cost efficiency. Azure Monitor can be used to view function app metrics with cost implications, while tools like Datadog Cloud Cost Management provide a holistic view of total cloud spend. Azure recommends following design principles such as avoiding long-running functions, writing stateless and idempotent functions, and minimizing the likelihood of errors and crashes through robust error handling and input validation. These practices help optimize function performance, reduce costs, and improve scalability and reliability. Securing Azure Functions is crucial to prevent data breaches, maintain compliance with regulatory requirements, and protect the availability of cloud services. Ensuring clients can only connect via HTTPS along with the latest TLS version, enforcing least privilege access control, and using role-based access control are some key strategies for securing Azure Functions. In summary, optimizing and securing Azure Functions involves choosing an appropriate hosting plan, monitoring cloud spend, following design principles, and implementing security measures to protect the application and its data.

Sensitive data is often unintentionally moved with financial services companies as they move to the cloud, posing costly breaches and governance, risk, and compliance (GRC) requirements. Datadog's Sensitive Data Scanner helps organizations identify sensitive data leaks and redact them at scale by discovering, classifying, and redacting sensitive data across logs, traces, RUM, and events. Financial services companies use Sensitive Data Scanner to detect credit cards, bank account numbers, and other personally identifiable information (PII) in observability data, classify matches against common compliance standards such as PCI-DSS and GDPR, and redact results to prevent sensitive data leaks into Datadog. They create multiple scanning groups to fine-tune rules for classifying their data and define actions Sensitive Data Scanner takes when it finds a match, including redacting, partially redacting, or hashing the matched value. By using Sensitive Data Scanner, financial services companies can improve accuracy by using keyword dictionaries and rule targeting, and follow up on sensitive data issues to comply with PCI-DSS obligations.

The text outlines how the Cloud SIEM team uses Datadog's Continuous Profiler, enhanced with custom context attributes, to improve the observability and performance of their applications that process large volumes of data. While the profiler provides insight into application performance in real-world conditions, challenges arise when shared methods across the codebase obscure detailed resource consumption analysis. Custom context attributes provide a solution by tagging code with additional runtime data, making profiles more actionable and aiding in identifying performance bottlenecks and mitigating incidents. The text also details how these attributes assist in isolating performance issues related to shared Java code and specific detection rules, ultimately streamlining troubleshooting processes. Additionally, it offers guidance on implementing custom context attributes in Java and Go applications to enrich profiling data and enhance troubleshooting capabilities without the need for complex custom metrics.

A meticulously crafted denial-of-service (DoS) attack was launched against Datadog's US5 website on April 19, 2024. The attackers targeted several Datadog regions, including a vulnerability in the Classic Application Load Balancer. The incident unfolded as the attackers sent OPTIONS requests with chunked payloads that were not properly handled by the load balancer, causing Envoy proxies to return HTTP 400 error responses. The attack was eventually mitigated through a combination of shifting traffic to an L4 load balancer, implementing custom request headers in Google Cloud Armor, and working with GCP to roll out a permanent fix for the vulnerability. The incident highlighted the need for cross-team collaboration, effective incident management processes, and tooling to detect and respond to complex DoS attacks.

Datadog has launched Security Inbox, a solution that provides security and engineering teams with valuable insights to mitigate security risks. It automatically organizes security risks into an actionable list for remediation, taking the guesswork out of addressing pressing security issues. The solution cuts through noise by analyzing severity levels, reviewing correlated risks, and understanding the impact of an issue on an environment. Security Inbox uses a multi-tiered system that prioritizes security risks based on their likelihood and impact, assigning severity levels using the Common Vulnerability Scoring System (CVSS) and the Datadog Security Scoring Framework. This allows teams to focus on the most critical risks with timely, accurate, and context-rich data, giving them the right context at the right time for protecting their environments.

The Datadog platform now partners with ITUnified to offer dbXplorer, a solution that simplifies database management and optimizes performance for Oracle databases. With dbXplorer, users can collect and visualize Oracle database telemetry in real-time, detect performance issues immediately, and maintain a high-performing database with optimal resource utilization. The integration includes 12 preconfigured alerts, including forecast, threshold, and anomaly detection, allowing users to promptly intervene whenever necessary. By using ITUnified's dbXplorer solution in Datadog, organizations can centralize the monitoring of multiple Oracle databases into one platform, ensuring efficient database performance, swift incident resolution, and a seamless user experience.

Monica Gavirangaswa is a key figure in highlighting the importance of observability challenges in GenAI applications, which often require multiple tools to monitor different aspects of the AI/ML tech stack. This can lead to blind spots and inefficiencies, ultimately affecting performance bottlenecks and user experience. Crest Data's integrations with Datadog Marketplace aim to address these pain points by unifying various aspects of GenAI observability within a single platform. Specifically, the Kong AI Gateway integration supports infrastructure monitoring, Together.ai integrates ML job performance monitoring, and TruLens Eval provides LLM app evaluation capabilities. These integrations enable teams to improve their AI infrastructure monitoring, ML job performance monitoring, and LLM app evaluation, ultimately overcoming fragmented AI monitoring challenges and achieving better performance in GenAI applications.

Datadog has been named a Leader in the 2024 Gartner Magic Quadrant for Observability Platforms for the fourth consecutive year, reflecting its commitment to solving customers' most sophisticated challenges. The company's unified observability and security platform breaks down silos among IT teams, enabling them to collaborate more effectively. Datadog's platform provides tools such as APM and Service Catalog to simplify microservice monitoring, Watchdog to speed up root cause analysis, and Workflow Automation to accelerate remediation of performance and security issues. The company believes that customer feedback is essential in analyst reports, showcasing how it supports customers in their day-to-day lives, allowing them to accomplish cloud migration and digital transformation initiatives successfully, securely, and cost-effectively.

Microsoft Fabric is Microsoft's unified platform for data analytics, integrating key Azure products like Azure Data Factory, Synapse, and Power BI. Datadog has integrated with Microsoft Fabric to enable customers to monitor SQL Pools, Big Data Pools, Synapse Workspaces, and SQL streaming assets within the Fabric platform. The integration provides out-of-the-box metrics, dashboard, and monitors to track asset health, activity, and performance, allowing users to identify issues and take corrective action. With this integration, Datadog supports Microsoft's fully vertical data analytics solution for generative AI and other big data use cases, providing a one-stop shop for data analysis and insights across various levels of expertise within an organization.

Anthropic, an AI research and development company, focuses on building reliable and safe AI systems, with its flagship product, Claude, being a notable language model excelling in natural language processing and reasoning. To enhance the management of Anthropic's AI applications, Datadog has introduced a native integration with Anthropic and LLM Observability, providing tools to monitor, troubleshoot, and secure these applications. This integration offers real-time metrics, detailed tracing capabilities, and quality checks to ensure performance, safety, and cost efficiency, making it easier for developers to handle complex use cases. LLM Observability also provides built-in evaluations for quality and safety, helping maintain ethical AI standards. Additionally, it helps in identifying errors and inefficiencies in AI application workflows, ensuring secure and effective usage of AI technologies. This service, now available to all Datadog customers, offers a comprehensive solution to manage AI applications effectively.

Datadog is a community-driven platform built by engineers and SREs, and it's celebrating its community with the upcoming Datadog Summit in Tokyo, Japan on October 16. The summit will feature customer talks and panels, as well as hands-on activities like workshops and AWS GameDay, where attendees can learn from experts and network with other users. The event is free, but space is limited, so it's recommended to RSVP now to reserve a seat. The summit will showcase how companies are using Datadog to develop innovative solutions to technical challenges, such as Degica enriching their alerting system and NTT Docomo evolving their products based on monitoring data. Attendees can expect to learn practical examples and experience with observability and monitoring tools through workshops, including introductory courses for SREs, intermediate courses on synthetic and real user monitoring, and beginner-level courses on APM and distributed tracing. The event also includes opportunities for community building, networking, and a gamified learning experience through AWS GameDay.

The text discusses the integration of Datadog Cloud SIEM with Akamai's Zero Trust and Application Security solutions. This integration aims to enhance security for users, devices, and data across an organization by securing access to the corporate network, SaaS applications, email, and other resources. The partnership enables organizations to increase their overall security posture, reduce risk, and improve operational efficiency and effectiveness of security operations teams. With this integration, Datadog Cloud SIEM provides several capabilities that help analyze Akamai logs, create custom detection rules, build dashboards for a shared view into security posture, and customize response workflows with Workflow Automation and Case Management.

The integration between Datadog and ServiceNow’s Configuration Management Database (CMDB) allows organizations to enhance their IT management processes by bridging data silos and providing critical context for monitoring and incident response. This integration enables DevOps and SRE teams to access host, network device, and service metadata, which can be ingested into Datadog as tags, facilitating more efficient triage and remediation of issues. By leveraging ServiceNow CMDB data, users can enrich logs and events in Datadog, improving the correlation of infrastructure components and identifying dependencies that may be impacted by performance issues. This collaboration not only streamlines access to IT management data within Datadog’s observability platform but also supports better decision-making by offering a comprehensive view of services' health and status.

Software ecosystems are becoming increasingly complex and fragmented, making it difficult for organizations to manage the thousands of interdependencies within their environments. To address this issue, Datadog has introduced Service Catalog schema v3.0 in private beta, which provides teams with an improved framework for modeling their software environments. The new version allows users to manually supplement and override automatically derived data, define a range of entity types, map relationships, and specify multi-ownership. Additionally, the IDE plugin helps developers create and validate their entity definitions, while Software Templates automate new entity setup. These enhancements enable teams to gain a clearer view of their tech stacks, improve collaboration during incidents, and more easily communicate architecture knowledge to new team members.