Tips for running Temporal on Kubernetes

Kubernetes serves as the foundational platform for deploying and managing cloud-native applications, excelling in maintaining application topology and managing infrastructure-level tasks but not handling internal application processes. To address this gap, Temporal offers a programming model that automatically manages state persistence, retries, and recovery, ensuring reliable workflow execution even amidst infrastructure instability. Deploying Temporal on Kubernetes enhances operational resilience by combining Kubernetes' infrastructure management with Temporal's application logic consistency. The use of official Helm charts simplifies the deployment of Temporal Server components on Kubernetes, while practical tips emphasize the importance of configuring external dependencies, managing upgrades, and securing the Temporal Web UI. Temporal Workers, which execute workflow and activity code, benefit from Kubernetes' capabilities for automatic restarts, horizontal scaling, and seamless deployments, with specific recommendations for containerization and deployment strategies. Effective autoscaling of Temporal Workers involves using task queue backlog metrics rather than traditional CPU and memory metrics, with KEDA-based autoscaling recommended for handling variable workloads efficiently. Capacity planning and tuning for production require iterative testing and adjustments to accommodate specific workload patterns, with attention to issues like CPU throttling that can affect performance. By aligning GOMAXPROCS with Pod CPU limits, users can prevent throttling and maintain consistent request latencies. Overall, deploying Temporal on Kubernetes provides a robust framework for scalable and reliable application operations, with additional support available through Temporal Cloud for those who prefer to focus on application development rather than cluster management.