Google Unveils Ironwood: 7th Gen TPU for Enhanced AI Inference

Google has introduced its seventh-generation Tensor Processing Unit (TPU) named Ironwood, which is specifically designed for inference workloads and marks a substantial advancement in AI computational power. Capable of scaling up to 9,216 chips, Ironwood offers an impressive 42.5 Exaflops of compute power, surpassing the world's largest supercomputer, El Capitan. It supports advanced AI models like Large Language Models and Mixture of Experts, emphasizing efficient data movement and latency reduction. Key features include 192 GB of High Bandwidth Memory per chip and a significant improvement in bandwidth and inter-chip communication. As part of Google's AI Hypercomputer architecture, Ironwood integrates with frameworks such as Vertex AI and Pathways, enhancing AI training and inference capabilities. Configurations include a smaller 256-chip pod and a larger 9,216-chip pod, with each chip supporting FP8 calculations for improved training throughput. Ironwood's power efficiency is doubled compared to its predecessor, Trillium, making it a leading option for enterprise AI applications within Google's AI ecosystem.