February 2025 Summaries

The biotech company Synlico has won a Golden Ticket from Lambda, which grants them six months of free access to an NVIDIA GH200 GPU. This prize is meant to help Synlico crack the code on diseases like cancer by understanding the true cause-and-effect relationships behind gene interactions. Synlico uses AI combined with causal discovery to navigate these pathways and steer drug development in the right direction. Their goal is to develop innovative therapies for oncology and other diseases, leveraging advanced AI technologies. With this prize, Synlico aims to accelerate their research and reduce computational bottlenecks, using the GPU instance to conduct testing faster and more cost-effectively. The company plans to release a large-scale single cell transcriptomics database with 57 million cells and 12,000 patient samples, which will be expanded in the future.

Lambda is launching a new Security page to provide information on its data safeguarding practices and secure cloud environment. The page will feature a SOC 2 Type II report, which is a testament to Lambda's rigorous auditing process ensuring high security standards. Users can also find avenues to report discovered vulnerabilities and access the company's Privacy Policy on the page. The goal of the Security page is to demonstrate Lambda's commitment to protecting users' data and workloads.

DeepSeek-R1 and v3 are being served on NVIDIA GH200 Grace Hopper Superchip instances, which provide a high throughput of 400 tokens per second. This is made possible by using 12 or 16 GPUs, depending on the required throughput. The model vLLM works better than Aphrodite for DeepSeek right now, and an update has improved its inference speed by roughly 40%. A script is provided to create instances, set up NFS caching, install Python 3.11, create a virtual environment, download models, install VLLM, and serve the model using ray. The guide includes a video showing inference speed with 64 parallel queries.

Lambda has raised $480M in funding to expand its AI cloud platform, which will enable faster and easier model training and deployment for developers. The company aims to build more software tools and deploy more GPUs to meet growing customer demand. With this investment, Lambda is well-positioned to power the impact of AI across various industries and sectors, with a focus on democratizing access to high-quality models through open-source reasoning.

The NVIDIA GH200 Grace Hopper Superchip is a unique datacenter hardware offering that combines an NVIDIA Hopper GPU with an ARM CPU via NVLink-C2C. This architecture promises to be promising for AI inference workloads requiring large KV cache allocations, thanks to the high-speed interconnect between the CPU and GPU. The GH200 has a higher memory bandwidth than the H100 GPU, improving generation speeds, while also offering advantages over H100 GPUs in prefill, such as offloading KV cache to abundant CPU memory. In experiments with Llama 3.3 70B on a single 96GB GH200 Superchip, the results show that the GH200 outperformed the H100 GPU by 32%, mainly due to the larger KV cache access. This suggests that the GH200 is an interesting processor for high-performance inference and model serving, particularly for large models that wouldn't fit on a standalone GPU with similar VRAM profiles.