November 2023 Summaries

Nebius AI has recently launched its cloud platform, offering a range of resources and insights into its capabilities and hardware. The platform includes walkthroughs and tutorials, such as setting up Kubernetes clusters for autoscaling and deploying PyTorch machine learning workloads. The platform's significance for ML/AI engineers is highlighted through its reliance on Kubernetes, with a blog post detailing its features and integration with tools like Terraform. Additionally, Nebius AI explores its hardware capabilities, showcasing the ISEG supercomputer's ranking and the company's focus on power-efficient data centers in Finland. The platform also provides guidance on selecting the right GPUs for specific workloads, distinguishing between options like NVIDIA’s H100 and A100 cards. Nebius AI is committed to delivering ongoing educational content through various formats to enhance user engagement and knowledge.

Igor, the Technical Product Manager for IaaS at Nebius AI, provides an in-depth exploration of NVIDIA's popular GPU chips, including the H100, L4, L40, and A100, discussing their relevance and optimization for transformer neural networks. These GPUs, built on Hopper, Ada Lovelace, Ampere, and Volta microarchitectures, are engineered to support transformer models, which have become the industry standard due to their superior performance in pre-training and fine-tuning tasks. Igor highlights the significance of numerical precision, particularly the FP8 format, in enhancing GPU performance for machine learning tasks, enabling models to be trained and inferred more efficiently. He also touches upon the different types of GPU cores, such as CUDA, Tensor, and RT cores, and their specific applications in deep learning and graphics. The discussion extends to the considerations for choosing between different GPU models based on specific use cases, such as machine learning, high-performance computing, and graphics, emphasizing the balance between cost, performance, and scalability. Nebius AI utilizes these NVIDIA GPUs in their data center in Finland, providing a platform for training, inference, and fine-tuning machine learning models.

In the dynamic realm of machine learning, infrastructure serves as a crucial backbone, facilitating seamless algorithm execution and model training. The text highlights the pivotal roles of Kubernetes and Terraform in orchestrating and managing this infrastructure, particularly when dealing with complex, GPU-intensive tasks. Kubernetes excels in autoscaling and orchestrating clusters, streamlining the deployment and maintenance of machine learning environments by automating tasks such as GPU driver installation and node management. The managed service variants of Kubernetes, like those offered by Nebius AI, further alleviate the burdens of setup and monitoring, enhancing efficiency and reducing administrative workload. Terraform complements Kubernetes by enabling infrastructure as code, allowing for reproducible and scalable deployments across different environments, thereby minimizing vendor lock-in and simplifying migrations. Together, these tools empower machine learning engineers to focus on innovation and algorithm development rather than infrastructure complexities.

The development of machine learning models for intelligent products and services relies heavily on powerful graphics processing units (GPUs), such as NVIDIA's H100, due to their ability to handle parallel processing tasks efficiently. Originally designed for the gaming industry's 3D demands, GPUs have proven indispensable for AI tasks by enabling the simultaneous calculation of large datasets split across numerous cores. The introduction of technologies like NVLink and RDMA has allowed for improved interconnectivity and data transfer between GPUs, facilitating the creation of powerful clusters essential for AI and machine learning (ML) applications. The text discusses the challenges and advancements in designing hardware tailored for ML, highlighting the transition from PCIe to SXM formats and the necessity for custom server designs to maximize GPU potential. Companies like Nebius are pioneering in deploying advanced GPU setups, such as the H100 SXM5, with tailored solutions for both training and inference tasks, ensuring rapid scalability and adaptability to evolving technological demands. This approach enables efficient data processing and model training, positioning them at the forefront of GPU cloud solutions by anticipating future needs and collaborating closely with manufacturers.

Starting November 1, users can explore detailed information about the products and pricing of Nebius AI's platform, which includes high-performance GPUs like the NVIDIA® H100 SXM5, hosted in a Finnish data center without waiting lists. The platform offers a comprehensive suite of services such as Compute Cloud, Managed Kubernetes, Object Storage, and network drives, along with essential security services and a Marketplace. The H100 SXM5 GPUs are interconnected with 3.2 Tb/s InfiniBand, making them suitable for LLM or generative AI models, and are available at rates starting from $4.85/hour under a pay-as-you-go model, with potential reductions to $3.15/hour for long-term commitments. Nebius AI claims that users can save at least 50% on GPU compute costs compared to major public cloud providers. To ensure smooth adoption, a dedicated engineer is provided to assist with infrastructure optimization and Kubernetes deployment, with access granted upon confirmation by the Nebius AI team.