HeyGen x Google Cloud: Bringing Avatar IV to TPUs

HeyGen and Google Cloud optimized Avatar IV, an AI talking-head video system with more than 18 billion parameters, for an eight-chip Google Cloud Trillium TPU host, achieving a 1.86× reduction in per-chunk rendering time while maintaining output-quality requirements. The pipeline uses two diffusion transformers and a VAE decoder, with model weights sharded across chips and sequence parallelism used to handle video workloads within memory limits. Key improvements included pipelining sequence-parallel all-to-all communications so transfers could overlap with attention computation, redesigning sparse attention kernels around frame-aligned blocks to eliminate masking, padding, and extra passes, and replacing the serial running-maximum operation in many attention softmax calculations with mathematically bounded precomputed values. The teams also improved compiler interaction by aligning kernel output layouts with collective inputs, setting specialized scheduling options, and supplying kernel cost estimates to XLA. All changes underwent byte-identical or tightly controlled similarity testing, and an otherwise faster lower-precision residual-stream optimization was rejected because it reduced output quality; the resulting TPU pipeline reportedly approaches the performance of HeyGen’s eight-H100 deployment while offering up to 25% better cost efficiency per generated video minute.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.