NVIDIA GH200 Grace Hopper Superchips Now on Lambda and Available On-Demand

The NVIDIA GH200 Grace Hopper Superchip is a powerful and efficient accelerated computing platform available on AWS Lambda On-Demand. It features a 72-core NVIDIA Grace CPU with an NVIDIA H100 Tensor Core GPU, connected by a high-bandwidth NVLink-C2C interconnect, offering up to 900GB/s of total memory bandwidth. This results in faster time-to-first-token (TTFT) for models like Llama3 70B. The Superchip is designed for scientific HPC workloads and offers an optimal solution for simulation-intensive applications in fields like material science and fluid dynamics. It provides a high-performance, cost-effective solution for AI/ML and HPC teams, with the option to scale up to large clusters with up to 720 Grace CPUs and 960GB of H100 GPU memory.