March 2023 Summaries

In early April, Lambda will introduce NVIDIA H100 Tensor Core GPUs to its cloud platform, offering high-performance instances with the fastest GPU type on the market. The addition of these powerful GPUs will enable customers to accelerate their AI and machine learning workloads, particularly for large-scale training and inference tasks. This collaboration between Lambda and NVIDIA aims to support enterprises in their adoption of cloud-first AI strategies, driving long-term success across business environments. With the launch of NVIDIA H100 GPUs, Lambda is well-positioned to be one of the first to market with this technology, providing customers with a plug-and-play experience complete with the latest hardware, software, and technical support.

Today, Lambda announced the raise of $44 million in Series B funding to accelerate the growth of its AI cloud. The round was led by Mercato Partners and also saw participation from prominent investors such as Adam D'Angelo and Jeff Hammerbacher. With this funding, Lambda plans to deploy new H100 GPU capacity with high-speed network interconnects, develop and launch features that will make it the best cloud in the world for training AI, and continue to optimize its cloud for cost-effective AI compute. Building on its existing success, Lambda aims to build a cloud where training AI models is as easy as doing so on a laptop, regardless of the number of GPUs used.

The text explains how to use mpirun to launch an LLaMA inference job across multiple cloud instances. This allows for more memory-efficient model training and inference, even without a multi-GPU workstation or server. The process involves setting up a cluster of cloud instances with SSH key login, cloning the LLaMA repository, installing dependencies, and running a shell script to automate these steps. Once set up, users can launch interactive inference jobs using mpirun, which allows for faster inference speeds compared to other methods. The cost of running an LLaMA job on Lambda Cloud is also estimated, with costs varying depending on the model size and instance type. Overall, this tutorial provides a practical guide for deploying LLaMA on cloud infrastructure.