April 2025 Summaries

The newsletter highlights several significant updates and events from Temporal, including new integrations with Datadog and New Relic that enhance observability for Temporal Cloud, enabling better workflow monitoring. The latest KEDA version introduces a Temporal Worker scaler, facilitating dynamic scaling of workers based on task queue backlog in Kubernetes environments, while Task Queue Priority offers control over workflow execution order. Updates across various SDKs, such as the Go, Java, .NET, PHP, and Ruby SDKs, introduce features like Activity Pause and enhanced telemetry support. Temporal is also participating in events like Stripe Sessions, offering interactive demos and insights into workflow orchestration and application development. Additionally, the newsletter covers upcoming webinars on building AI agents with Temporal's Python SDK and a Temporal 101 session in Go, alongside invitations to engage with the community through programs like Meetup in a Box and highlights from recent community contributions.

Temporal, co-founded by Max Fateev and Samar Abbas, is an open-source orchestration tool designed to simplify workflow management for developers by allowing them to focus on business logic without worrying about failure recovery, state, or time. Drawing on their experiences at Amazon, Microsoft, and Uber, Fateev and Abbas developed Temporal to address the complexities of distributed systems and improve the developer experience. Temporal's core innovation, "Durable Execution," ensures the continuity of workflows even across failures, making it particularly appealing to financial services and other industries that require resilience and precision. The platform emphasizes security, with a secure-by-design approach that separates user applications from the backend orchestration cluster. Temporal's open-source nature under an MIT license encourages community-driven improvements and broad adoption, with an optional cloud-hosted service that offers managed infrastructure while allowing developers to retain control over their applications and data.

A new integration between Temporal Cloud and New Relic has been launched to enhance the monitoring and observability of Temporal Workflows, which are crucial in industries such as finance, AI, healthcare, and logistics. This integration allows engineering teams to incorporate Temporal Cloud metrics directly into New Relic dashboards for real-time insights into Workflow execution, system behavior, and service performance. The integration features live monitoring, pre-built dashboards, alert support, and anomaly detection, facilitating the identification of bottlenecks and ensuring reliable system operation at scale. It simplifies the setup process through the New Relic Flex integration, offering a seamless connection between Temporal workloads and existing monitoring practices. Prior to this, users had limited visibility into Temporal’s metrics, but now they can achieve a robust and production-ready monitoring experience with minimal setup, benefiting mission-critical systems and complex architectures.

Deciding how to structure Activities within Temporal Cloud involves balancing the trade-offs between using more or fewer Activities, each carrying implications for system performance, cost, and workflow manageability. More Activities can increase the complexity and resource usage of the Temporal service, while fewer Activities might complicate workflow design and limit the use of built-in tools like Retry and Timeout Policies. Best practices suggest fine-grained Activities for simplicity, maintainability, and observability, but there are scenarios where combining tasks into a single Activity makes sense, such as when utilizing specialized hardware, needing separate retry or timeout policies, or when dealing with financial costs of certain operations. Improved software quality and developer experience can be achieved by preferring more Activities, which enhance code clarity and enable easier debugging and maintenance. Ultimately, the decision to use more or fewer Activities should consider necessity, practicality, and preference, ensuring the workflow design meets specific use case requirements efficiently.

A new integration between Temporal Cloud and Datadog is set to enhance the monitoring and optimization of Temporal Workflows by providing pre-built metrics dashboards and simplified setup. This integration allows users to access a Temporal Cloud metrics dashboard within Datadog, offering insights into system health and Workflow efficiency with minimal effort. By using a crawler-based approach to fetch metrics directly from Temporal Cloud’s Prometheus endpoints via REST APIs, this integration eliminates the need for a PromQL-to-Datadog scraper, reducing complexity and maintenance. Additionally, the integration provides cost-saving opportunities as the Temporal Cloud metrics are classified as standard, avoiding extra charges typically associated with custom metrics ingestion. It includes out-of-the-box dashboards, proactive alerts for critical metrics, and granular insights tagged with attributes like namespace and operation type. Setting up the integration is straightforward, requiring configuration of the Temporal Cloud Prometheus endpoint URL within Datadog and secure authentication through certificate upload.

KEDA's latest version 2.17.0 introduces the Temporal Worker scaler, enabling event-driven autoscaling for Temporal workers in Kubernetes environments. This feature allows dynamic scaling of Temporal workers based on Task Queue backlogs, providing cost efficiency by reducing infrastructure costs during low workloads and maintaining high throughput during traffic spikes. The integration, developed with contributions from the Temporal community and KEDA maintainers, is ideal for unpredictable workloads such as e-commerce and IoT data processing. A demo, based on Temporal's Order Management System, showcases the auto-scaling capabilities by adjusting Temporal Worker pods in real-time according to Task Queue activity, ensuring efficient workflow processing. The demo environment is available with necessary components, and the initiative highlights community contributions from individuals who played significant roles in its development, testing, and implementation.

Developers often face challenges when orchestrating complex workflows in distributed systems, which require managing long-running processes, coordinating asynchronous steps, and ensuring reliability amidst failures. Traditional solutions like explicit state machines can become complex and prone to errors as systems expand. Temporal offers a more developer-friendly approach by providing a durable execution model that abstracts the intricacies of distributed workflows, allowing developers to focus on business logic rather than infrastructure. Temporal enables developers to write straightforward code that functions as a state machine without the need for explicit state tracking, offering built-in retries, timeouts, and enhanced visibility for debugging. By automatically handling state persistence and recovery, Temporal simplifies the development of robust, fault-tolerant applications, reducing the need for manual failure handling and enhancing productivity. This approach results in cleaner, more maintainable code, as Temporal manages the underlying complexities of distributed workflow orchestration, making it a compelling alternative to traditional state machine implementations.

Mediaset, a major European broadcaster, transformed its complex and sluggish data infrastructure into a modern, efficient data stack using Temporal and Bauplan, drastically improving the performance and maintainability of their systems. Previously reliant on a cumbersome setup involving multiple AWS services like EMR, Glue, Athena, Redshift, and Airflow, Mediaset faced challenges with slow feedback loops and significant data processing delays, exemplified by a news dashboard that took an hour to refresh. By transitioning to Temporal for workflow orchestration and Bauplan for serverless data management, Mediaset achieved significant improvements, reducing data refresh times from an hour to five minutes and shortening development cycles from three months to six weeks. This reimagined architecture not only simplified infrastructure by reducing reliance on numerous AWS services but also enabled rapid iteration and resilient workflows, allowing Mediaset to focus on advanced applications such as AI-driven content summarization and real-time ad optimization.

Temporal Cloud for Startups aims to assist early-stage companies in accelerating their development processes without the burden of managing backend systems by offering a streamlined solution through its partnership with Google Cloud for Startups. This collaboration offers members of the Google for Startups Cloud Program $12,000 in free Temporal Cloud credits over 12 months, facilitating confident scaling and efficiency optimization. The plan includes fully managed Temporal Cloud services, eliminating concerns about server management, scaling, or uptime, and provides business-level support such as rapid response times, workflow troubleshooting, and SDK implementation guidance. Eligible startups need to be members of the Google Cloud for Startups program, maintain a Temporal Cloud account, run their first workflow, and have secured funding.

Temporal's Workflow management framework, in collaboration with AWS infrastructure, offers a robust solution for building resilient distributed systems by simplifying the creation of fault-tolerant applications. By utilizing Temporal’s TypeScript SDK, developers can efficiently manage complex processes such as e-commerce order fulfillment through clear Workflow steps, state management, error handling, and Workflow monitoring. The framework handles worker outages seamlessly, enabling continuous execution by preserving Workflow states and offering smart error handling with automatic retries for intermittent failures. Temporal also supports integrating human decision-making into Workflows through signals, allowing for pauses and resumptions based on external inputs, with timeouts to prevent indefinite waits. The comprehensive web UI aids in monitoring and debugging by providing transparency into Workflow executions. Temporal Cloud, hosted on AWS, further enhances this capability by equipping developers with tools that simplify state management, error handling, and manual interactions, ultimately improving system reliability and reducing complexity.