May 2020 Summaries

The NVIDIA A100 GPU is expected to provide significant performance gains for deep learning applications, particularly those using FP16 Tensor Cores. When compared to the V100 GPU, the A100 is expected to offer a 1.95x to 2.5x speedup for language model training, with actual performance potentially exceeding 18.1 TFLOPS. The A100's design focuses on maximizing deep learning performance through increased power budget allocation to FP16, Tensor Cores, and other features like sparsity and TF32. The DGX A100 server, featuring up to 8x A100 GPUs, offers superior node-to-node communication bandwidth compared to the DGX-1 or Lambda Hyperplane-8 V100, which may provide better cluster scaling performance. The A100's design also enables a near doubling of FP16 efficiency and represents a significant jump in process node from TSMC 12nm to TSMC 7nm.

AWS Lambda Cloud now offers an affordable 8-GPU V100 instance type for deep learning and machine learning workloads, priced at $4.40 per hour. This instance provides over 2x more compute power per dollar compared to comparable on-demand instances from other cloud providers. The new instance includes 8x NVIDIA Tensor Core V100 SXM2 GPUs with NVLink, 92 vCPUs, 448 GB system RAM, and 6 TB of temporary local storage. NVLink allows for faster communication between GPUs, but still requires distributed training workloads. The Lambda Cloud instance costs 51% less than comparable on-demand instances from other cloud providers when running full precision models. When combined with the included 6 TB of temporary local storage, the instance provides over 2x the compute power per dollar, making it a cost-effective option for deep learning and machine learning workloads.

You can launch a Lambda Cloud GPU instance and use SSH to log in, which offers benefits such as copying files from your local machine and using a local terminal. To spin up an instance, you'll need to add an SSH key if you don't already have one. If you do have an existing key, you can use it by copying the public key into the text field on the dashboard. Alternatively, you can create a new key locally using `ssh-keygen` or let Lambda Cloud create one for you. You'll also need to set up your local SSH directory and move the downloaded key into it with proper permissions. Once everything is set up, you can spin up an instance and log in using the SSH command. Note that instances are billed by the hour and do not offer persistent storage, so any data will be deleted when the instance is terminated.

This guide provides a comprehensive overview of how to transfer data to Lambda Cloud GPU instances, including copying files from various sources such as local desktops or laptops, publicly accessible URLs and cloud storage buckets, private S3, Azure, or Google Cloud buckets. The guide covers the use of SCP, Wget, AWS CLI, gcloud, gsutil, and AzCopy for data transfer. It also includes instructions on how to decompress compressed files and move data sets to new directories. Finally, it offers final thoughts on using Lambda Cloud instances and provides contact information for feedback or questions.