December 2022 Summaries

Cansu Berkem, Group Product Manager at Datadog, shares her experience over the last year, highlighting her work on integrating Cloudcraft with Datadog's products to improve architectural visualization of customers' infrastructure. She discusses her day-to-day responsibilities as a PM, emphasizing the importance of customer research and collaboration with engineering and design teams. Product Managers at Datadog play a crucial role in driving product roadmaps, ensuring growth and customer satisfaction while staying competitive in the market. The company values diversity and fosters a supportive environment for its PMs to share ideas and learn from each other. Cansu believes now is an excellent time to join the Datadog Product Management team due to unique challenges, exceptional talent, and a strong focus on growth, innovation, and culture.

Cansu Berkem, a Group Product Manager at Datadog, has been leading her team and working on various exciting projects, including the acquisition and integration of Cloudcraft. As a PM at Datadog, she has had the opportunity to present in keynotes, travel to conferences, and collaborate with engineering teams to improve products. A typical day for Cansu involves wearing multiple hats and switching between different products, requiring her to balance customer research, design, and delivery of best-in-class products. The Product Management team at Datadog has a strong culture that values growth, innovation, and diversity, making it an exciting time to join the team.

The text explores best practices for implementing continuous testing (CT) across the software development life cycle, emphasizing the importance of testing at multiple stages to catch issues early and ensure application functionality for customers. It discusses strategies such as creating reusable tests, running tests in parallel to reduce execution times, and integrating tests into CI/CD pipelines to enhance visibility and efficiency. Datadog Continuous Testing is highlighted as a tool that supports these practices by allowing tests to be run across different environments, optimizing parallel execution, and providing end-to-end visibility in development workflows through integration with popular CI/CD providers. The text underscores the role of shift-left testing in improving test results and application performance by giving developers earlier insights into potential issues.

The text discusses best practices for maintaining test suites in application workflows, emphasizing the importance of reducing test flakiness and false positives, making tests easy to understand, and leveraging a unique test suite across different environments. It highlights challenges like evolving UIs and test duplication, which complicate test maintenance, and suggests solutions such as using reliable locators, fine-tuning alerts, and consolidating workflow steps. The text also details how Datadog can aid in maintaining robust test suites by using multi-locator algorithms to adapt to UI changes and integrating tests with continuous integration (CI) workflows to ensure consistent test coverage and collaboration across teams. By implementing these strategies, teams can maintain efficient and reliable test suites that enhance troubleshooting and decrease maintenance efforts.

Testing plays a crucial role in the development process as it helps detect broken functionality before customers do. Two main categories of testing are pre-production and post-production. The shift-left model, which involves verifying functionality earlier in the development cycle, is gaining popularity to find and fix issues faster. This approach can incorporate various types of tests, including unit, API, and browser tests. Creating meaningful test coverage requires identifying commonly used application workflows and prioritizing them over less popular ones. Tools like real user monitoring (RUM) can help determine the most important user journeys to include in end-to-end (E2E) test suites. It is also essential to create tests for flows that are not frequently used but are still critical to users' experience. To build efficient and meaningful tests, teams should follow best practices such as breaking workflows into smaller, focused tests, creating meaningful assertions, building adaptable tests, and maintaining the state of applications with idempotent tests. Additionally, organizing test suites by testing environments and using metadata to categorize tests can help manage large test suites effectively. Finally, creating notifications that enable teams to respond faster is crucial for addressing issues promptly. Datadog's CI integration and built-in HTTP request steps can further streamline the process of testing code before it reaches staging or production environments.

The text discusses the importance of comprehensive testing strategies in ensuring high usability and user satisfaction for web applications. It introduces Datadog Synthetic Monitoring and RUM as tools that enable teams to build their testing suites based on real user data, addressing challenges such as assessing test effectiveness and identifying untested workflows. The new Test Coverage page provides crucial insights to optimize the test suite by understanding overall test coverage, identifying untested actions, and refining existing tests. This feature is now generally available for all Datadog RUM and Synthetic Monitoring customers.

Large IoT environments consist of multiple layers of disparate devices that require secure and reliable data transfer between them. HiveMQ is an open source MQTT broker designed for enterprise-scale IoT fleets, providing visibility into each layer of the IoT environment to quickly identify problems with deployment. Datadog's HiveMQ integration collects metrics and event logs from HiveMQ, offering out-of-the-box dashboards that help visualize key data such as connections, disconnects, and message throughput. To provide visibility into IoT applications, HiveMQ's Distributed Tracing Extension enables users to instrument their workloads to generate OpenTelemetry-formatted distributed traces, which track application processes from end to end throughout the environment. By sending these traces to Datadog, users can use Application Performance Monitoring (APM) for more effective troubleshooting and root cause analysis. HiveMQ's extension and OTel support provide enhanced IoT observability across the environment, allowing users to monitor OpenTelemetry traces from IoT applications connected to the HiveMQ broker using Datadog APM.

HiveMQ is an open source MQTT broker that provides secure and reliable data transfer between IoT devices and downstream applications, enabling visibility into the health of IoT infrastructure. Datadog's HiveMQ integration collects metrics and event logs from HiveMQ, providing out-of-the-box dashboards for visualizing and alerting on key data. The Distributed Tracing Extension enables users to instrument their workloads to generate OpenTelemetry-formatted distributed traces, which can be sent to Datadog Application Performance Monitoring (APM) for enhanced visibility into application performance. With HiveMQ's OTel-instrumented traces and Datadog's support for OpenTelemetry, users can quickly spot issues in IoT application performance, identify root causes, remediate user experience problems, and understand how their applications interact with each other within their broader infrastructure.

This article discusses the importance of protecting applications in cloud environments and provides best practices to achieve this. It highlights the complexities of modern application architectures, such as microservices, and how they can be vulnerable to various security threats. The article emphasizes staying up-to-date on the latest threats, using threat modeling to identify vulnerabilities in an application's design, implementing validation and access controls at the application level, and creating an effective monitoring strategy. These practices help organizations build a comprehensive defense-in-depth strategy for their cloud applications.

Staying up to date on the latest threats is crucial for protecting cloud-based applications from vulnerabilities and attacks. Organizations must leverage threat modeling to surface vulnerabilities in an application's design, implement validation and access controls at the application level, and create an effective monitoring strategy that includes detecting malicious activity at the network, endpoint, and code layers. This enables organizations to build a defense-in-depth strategy that complements their existing security measures, including endpoint and network perimeter security. By following best practices such as input validation, authentication and authorization, and software composition analysis, organizations can protect their cloud-based applications from various types of attacks and threats.

The text discusses the growing adoption of serverless architectures, highlighting the security challenges they pose due to their distributed and stateless nature, which complicates protection against application-level attacks like SQL injection and cross-site scripting. It introduces Datadog's Application Security Management (ASM) as a solution for enhancing security in serverless applications, specifically on AWS Lambda, by detecting threats through monitoring requests and generating security signals. ASM utilizes distributed tracing to investigate attack scopes and facilitates responses within existing workflows, offering features like Open Source Vulnerability Detection for third-party dependencies. The text emphasizes Datadog ASM's capability to provide insights into attacks and aid in quick responses, allowing developers to focus more on innovation while maintaining robust security. Datadog ASM is available for AWS Lambda, with Open Source Vulnerability Detection in public beta, and offers a free trial for new users.

Serverless architectures are increasingly popular as they simplify scalability and infrastructure management, with significant adoption seen across major cloud platforms such as Azure, Google Cloud, and AWS. However, securing serverless applications remains a challenge due to their distributed nature and stateless components, which require security measures at multiple layers, including both code and invocation levels. Datadog App and API Protection (AAP) addresses these challenges by providing real-time monitoring of serverless applications, particularly AWS Lambda functions, to detect and respond to security threats like SQL injection and Cross-Site Scripting (XSS) attacks. AAP uses Datadog's tracing library to monitor application traffic without additional agents, generating security signals for detected threats and enabling users to quickly investigate attack details through distributed tracing. This feature allows for efficient threat response and remediation within existing workflows, integrating easily with tools such as Slack and Jira. Datadog AAP's out-of-the-box detection rules and customizable options help organizations maintain robust security postures, allowing them to focus on development and innovation while ensuring the protection of their serverless environments.

In 2021, Datadog introduced CI Visibility to provide granular performance metrics for each individual pipeline, allowing users to monitor build duration and related telemetry across all recent commits. However, the high frequency of commits and complexity of CI deployments in modern applications make it difficult to keep a constant eye on the health of your pipelines. To address this issue, Datadog has introduced CI pipeline monitors that enable users to set alerts on key performance metrics for their pipelines. These monitors notify users whenever pipeline performance degrades in a predefined manner, such as when builds extend beyond an acceptable duration or fail at an unacceptable rate. The granular alerts across all pipelines and stages help pinpoint the source of bottlenecks and failures more easily, while diagnostic insights into degrading pipelines enable users to quickly identify what broke them and how they can prevent it from happening again.

Datadog has introduced CI pipeline monitors to provide granular performance metrics and alerts for each individual pipeline, enabling teams to monitor build duration and related telemetry across all recent commits. These monitors allow users to set alerts on key performance metrics, providing visibility into the health of their pipelines and enabling them to respond quickly to issues. With pipeline monitors, users can receive notifications when pipeline performance degrades, allowing them to troubleshoot and resolve problems promptly, reducing the risk of lengthy interruptions and improving overall development workflow efficiency.

At AWS re:Invent 2022, three main themes emerged from the conference announcements and sessions: using data and observability to improve control of systems; strengthening security by gathering intelligence to augment mitigation; and increasing performance to deliver results faster and save costs. The event featured new product releases, technical sessions, networking opportunities, and discussions on leveraging data for better business decision-making and dynamically controlling digital stacks. AWS CEO Adam Selipsky highlighted the exponential growth of data and its role in innovation. New features such as AWS Control Tower Account Factory Customization and Amazon Security Lake were announced to enhance security and observability. Performance improvements included new EC2 instance types, with Graviton-powered instances offering significant cost savings.

AWS re:Invent, a post-Thanksgiving tradition for some at Datadog, featured new product announcements, technical sessions, networking, and fun. The conference highlighted three themes: using data and observability to improve control of systems, strengthening security by gathering intelligence to augment mitigation, and increasing performance to deliver results faster and save costs. AWS announced several new features, including AWS Control Tower Account Factory Customization, Universal Service Monitoring (USM), Amazon Security Lake, and Graviton-powered EC2 instance types. These announcements were met with excitement and curiosity among attendees, who are eager to adopt these new tools and technologies to improve their systems' performance, security, and control.