September 2025 Summaries

Nebius has achieved NVIDIA Exemplar Status as one of the first NVIDIA Cloud Partners on NVIDIA H200 GPUs for training workloads, demonstrating its capability to meet NVIDIA's high standards for performance, resilience, and scalability in AI infrastructure. This recognition addresses the challenges of ensuring consistent workload performance and predictable costs across clouds, which are critical for modern AI workloads that require scaling to thousands of interconnected GPUs. Unlike hyperscale clouds, which can be costly and inconsistent, or bare-metal GPU providers, which lack flexibility, Nebius offers a vertically-integrated stack with custom NVIDIA-accelerated servers in energy-efficient data centers, providing full control over quality assurance and performance tuning. Nebius' infrastructure demonstrated over 97% of the performance benchmark on NVIDIA H200 GPU clusters, optimized networking stack scheduling, and achieved high reliability with 167,000 GPU hours MTBF on a 3,000-GPU cluster. The company supports a true cloud experience for AI/ML developers, offering managed Kubernetes, built-in AI orchestration, and native AI/ML ecosystem options, along with responsive human support and flexible scaling capabilities to meet various customer demands. This status assures enterprises, AI labs, and startups that Nebius can provide reliable infrastructure for advancing AI projects.

Tokenization is a crucial preprocessing step in AI that allows language models to understand and generate human language by converting text into tokens, which are smaller, manageable pieces that can be mathematically processed. These tokens can be whole words, subwords, or characters, each mapped to a unique numerical ID, enabling models like GPT and BERT to interpret language effectively. Various tokenization methods, such as word, character, and subword tokenization, each have their advantages and limitations, impacting the model's efficiency, accuracy, and ability to handle unusual words. Tokenizers used in large language models (LLMs) are essential components, ensuring consistency between training and inference phases. Creating custom tokenizers can be beneficial for niche applications or low-resource languages, using tools like Hugging Face and SentencePiece to train and deploy tokenizers efficiently. The design of a tokenizer not only influences model performance but also affects computational costs and the potential introduction of biases, making it a foundational element in the development of robust and adaptable AI systems.

AI agents are revolutionizing customer support by providing 24/7 assistance, understanding urgency, and processing queries with company-specific details. This transformation is powered by large foundation models, and the article outlines how to build a multi-agent system using an open-source Agent Development Kit (ADK) and the Agent-to-Agent (A2A) protocol. The ADK, developed by Google, offers a modular and flexible framework for developing AI agents that can scale, test, and integrate into real-world systems. The A2A protocol enables communication and collaboration between AI agents, allowing them to solve complex problems collectively. The article details the development of a customer query routing and resolution system, where a coordinator agent orchestrates tasks among intake, resolution, and escalation agents based on the sentiment of customer messages. The project uses open-source models from Nebius AI Studio for various tasks, optimizing performance and cost. The system is demonstrated through a Streamlit-based UI, showcasing its potential for real-world applications like automated onboarding, internal helpdesk automation, and multilingual support.

Writer’s block is a common challenge for content teams, especially when needing to produce trending and company-aligned social media content daily. The article outlines a solution by building a content generator platform using Nebius AI Studio and Qdrant, which can be tailored to specific company needs. This platform employs Retrieval Augmented Generation (RAG), combining search and generation to create context-aware content. Users upload company data, which is processed into vector embeddings stored in Qdrant Cloud for semantic search. When content is requested, the system retrieves relevant context based on semantic similarity and utilizes the Llama-3.3-70B-Instruct model to generate personalized content suggestions. The platform features a React-based dashboard for content management, analytics, and history tracking, and it is designed to be scalable and efficient, allowing companies to enhance content creation with AI capabilities without managing complex infrastructure.

Model distillation is a technique where a smaller student model is trained to replicate a larger teacher model's performance by learning from the teacher's soft outputs, such as probability distributions, rather than just hard labels. This method is advantageous as it reduces memory usage, infrastructure costs, and latency without compromising accuracy, making it suitable for deploying specialized models in real-world systems. Distillation is particularly important for state-of-the-art large language models (LLMs) that are expensive and resource-intensive, as it allows for the creation of smaller, task-specific models that maintain a high level of performance. The process involves running both teacher and student models through extensive datasets, making GPU acceleration crucial for reducing training times and improving convergence. Model distillation enables the deployment of robust AI systems on constrained hardware and at a lower operational cost, facilitating the integration of advanced AI capabilities into production environments, edge devices, and budget-conscious applications.

On September 3, 2025, Nebius experienced a 1 hour and 45-minute service disruption in the us-central1 region due to a routing configuration conflict between network domains, which was triggered by a combination of a past routing policy optimization and a recent security infrastructure extension. This conflict led to a persistent routing loop that affected multiple customer-facing services and exposed cross-regional dependencies, amplifying the impact beyond the immediate region. Although other regions remained functional, inconsistent service experiences were reported as the load balancing system continued directing users to both functional and non-functional endpoints. The disruption impacted public API operations, console communication, virtual machine management, tenant registration, and developer tools targeting the affected region, while those targeting other regions experienced limited resource visibility. The root cause was traced to a latent configuration issue that was not detected during testing and a subsequent routing recomputation that perpetuated the loop. In response, Nebius has outlined improvements in network infrastructure and service architecture, such as implementing stricter routing controls, enhancing monitoring and alerting systems, reducing cross-regional dependencies, and improving failover mechanisms to prevent future incidents and minimize customer impact.

Jupyter Notebook serves as an interactive programming and data analysis tool, primarily used for machine learning tasks, enabling users to run code in various languages while embedding visual outputs, mathematical formulas, and narrative text in a single document. It connects to a kernel that allows for persistent session states, making it ideal for iterative experimentation, debugging, and hypothesis testing. This flexibility is particularly useful in machine learning workflows, where data preparation, model training, and result visualization occur within the same environment. Despite its effectiveness for prototyping and experimentation, Jupyter Notebooks are not designed for production deployment due to their lack of support for streaming workloads, automated deployment, and service-level fault tolerance. They often require conversion into more modular and testable code for integration into production pipelines. However, cloud platforms, like Nebius AI Cloud, enhance Jupyter's capabilities by offering preconfigured environments with GPU access and seamless integration into managed infrastructure, thus facilitating the transition from experimentation to deployment in AI workflows.

Nebius has achieved leading performance results in the MLPerf® Inference v5.1 benchmarks, showcasing its AI systems powered by NVIDIA's high-demand GPUs such as the GB200 NVL72, HGX B200, and HGX H200. These results highlight Nebius' improvements in inference performance, with significant gains in token throughput across various configurations, notably outperforming previous benchmarks with systems like the NVIDIA GB200 NVL72 for large foundational models such as Llama 2 70B and Llama 3.1 405B. The benchmarks underscore Nebius' capability to run AI workloads efficiently in virtualized environments without compromising performance, thanks to their engineering expertise in utilizing NVIDIA hardware and software. The company's achievements illustrate its commitment to providing high-performance, scalable AI infrastructure that remains competitive with top-tier industry standards, offering customers supercomputer-level performance and reliability with the flexibility of a hyperscaler.

The decision between using a single node or a cluster for AI workloads is influenced by factors such as model size, dataset size, budget, and operational needs. A single node, which consolidates all computation locally, is advantageous for early research, prototyping, and production tasks with moderate demands due to its simplicity, lower costs, and ease of management. However, as the need for scalability arises, clusters become essential, particularly for training large models, handling high-volume requests, and ensuring redundancy and uptime. Clusters distribute workloads across multiple nodes, allowing for faster training times and increased resource efficiency, though they come with higher operational overhead. The choice hinges on balancing speed, cost, and complexity, with hybrid and cloud-based strategies offering a flexible path to scaling. The infrastructure should adapt as project demands grow, starting with single-node setups for smaller tasks and transitioning to clusters for more extensive requirements, leveraging modern accelerators and cloud elasticity to optimize performance and cost-effectiveness.

NVIDIA Blackwell GPUs are now accessible as self-service clusters in Nebius AI Cloud, allowing users to leverage the latest accelerated computing technology with ease. Nebius has enhanced cluster reliability for AI workloads, and their recent whitepaper highlights liquid cooling as a significant trend in AI data centers. The SWE-rebench dataset, developed by Nebius's AI R&D team for evaluating code agents, has gained popularity on Hugging Face. Nebius also supports TrialHub in improving clinical trial efficiency and collaborates on BioHackCloud, a federated multi-cloud bioinformatics project. Additionally, Nebius has expanded its technical documentation, offering comprehensive guides on Kubernetes networking, Compute networking, object storage management, and observability, while also extending Managed PostgreSQL services to new regions.

A Virtual Private Cloud (VPC) represents a logically isolated segment within a cloud provider's infrastructure, enabling businesses to run applications securely with enhanced control over network configurations such as IP ranges, subnets, and security protocols. It mirrors the structure of traditional on-premises networks but benefits from cloud-native scalability and flexibility, making it particularly valuable for machine learning tasks that demand predictable, isolated environments. The VPC's key advantages, including security, flexibility, and scalability, stem from its ability to define access rules, segment environments, and seamlessly scale resources, which are crucial for managing sensitive data and ensuring compliance. With core elements like routing tables, security groups, and VPN connections, VPCs facilitate secure, efficient, and adaptable network management, serving as a foundation for hybrid cloud strategies and modern infrastructure needs. In essence, VPCs offer a blend of the control found in on-premises setups with the agility of cloud solutions, thus becoming a default choice for organizations prioritizing both growth and security.

The BioHackCloud project, a collaboration involving leading bioinformaticians and computational biologists from academia and industry, aims to advance the secure and responsible use of genomic data through a federated cloud platform based on the Global Alliance for Genomics and Health's (GA4GH) open framework. Enabled by Nebius AI Cloud and its partners, the project has made significant strides in safeguarding sensitive biomedical workflows with technologies like attested TLS (aTLS) and confidential computing, ensuring privacy and cross-cloud interoperability. Key achievements include a proof of concept for aTLS, which allows users to verify the trustworthiness of software in secure execution environments, and private LLM inference through an OpenAI-compatible API. The initiative also supports confidential task execution via ELIXIR TESK, a Kubernetes-native implementation for multi-cloud environments, and leverages Nebius' Slurm-on-Kubernetes for efficient GPU resource management. As part of the broader European Open Science Cloud (EOSC) and through efforts like the TRE Open Suite project, BioHackCloud aims to enhance privacy and security standards while enabling cross-border bioinformatics. Looking forward, the project plans to deepen its frameworks and showcase new attested TLS implementations during the 2025 ELIXIR BioHackathon, offering a sandbox for technological exploration in genomics, AI, and bioinformatics.

GPUs have become a crucial component for modern AI due to their ability to handle the parallel processing demands inherent in tasks like large language model training, image generation, and real-time inference. Their unique architecture allows them to perform many operations simultaneously, making them indispensable for deep learning frameworks such as TensorFlow and PyTorch. Access to GPU servers can be achieved through various models, including cloud GPU servers, dedicated GPU instances, bare-metal GPU servers, and hybrid or on-premises clusters. Each model offers different benefits, such as scalability, performance consistency, and control, depending on the specific requirements of AI workloads and organizational needs. These GPU servers are essential for training large models, executing inference tasks, and managing sensitive data, with the choice of model often determined by budget, workload type, and compliance considerations. Leveraging these different GPU access models enables teams to align their infrastructure with their performance, cost, and control objectives, ensuring efficient and scalable AI operations.