September 2023 Summaries

Persistent storage for Lambda Cloud is now available in most regions, allowing on-demand customers to persist files and data when using Lambda's compute. The service has exited beta and is no longer free, with storage costing $0.20/GB/month. Filesystems are not global and cannot be attached across regions, and there is no quota or limit to how much can be stored. Data in filesystems cannot be accessed without attaching to a VM, so it's recommended to download anticipated data before shutting down an instance.

The study explores whether a Large Language Model (LLM) like OpenAI's GPT can assist human readers in digesting scientific reviews by generating a "meta review" that provides an accept/reject recommendation along with the confidence level and explanation. The investigation uses the NeurIPS 2022 conference data, where each paper has 3 to 6 general reviews, along with a meta review giving the final decision based on these reviews. The results show that GPT is able to generate useful review summaries and "better than chance" accept/reject recommendations. However, it tends to lean towards acceptance and hesitates to recommend rejection. Guiding the AI via directive and indirective prompts improves its accuracy for rejected papers. The absence of reviewer ratings significantly affects AI accuracy, highlighting their importance. While AI holds promise in aiding peer reviews, challenges remain in refining its decision-making processes and addressing biases.

Lambda has launched a new Hyperplane server with NVIDIA H100 GPUs and AMD EPYC 9004 series CPUs, offering unprecedented performance, scalability, and security for every workload. The new server combines the fastest GPU type on the market with the world's best data center CPU to deliver enhanced performance per dollar. Lambda customers can now train and run inference faster with this new server, which is available in Cloud Clusters starting at $1.89 per H100 per hour, as well as in on-prem Hyperplane Servers and Clusters. The Hyperplane 8-H100 HGX Server features 8x NVIDIA H100 SXM Tensor Core GPUs, dual AMD EPYC 9004 CPUs with up to 256 cores total, and up to 6TB of DDR5 system memory and 480TB of local NVMe storage. This new server enables faster training times and better performance per dollar on Generative AI models and other distributed training applications.