August 2025 Summaries

This series of blog posts explores the process of writing high-performance GPU kernels on NVIDIA's Blackwell architecture, aiming to achieve performance competitive with NVIDIA's cuBLAS library. The series serves as a reference for optimizing Blackwell GPUs, filling a gap in existing documentation for this new architecture. In Part 1, the series introduces the importance of matrix multiplication (matmul) for large language models (LLMs) and outlines a simple GPU implementation in Mojo. It highlights the role of GPUs in executing data-parallel operations like matmul, which constitutes a significant portion of LLMs' runtime. The series also delves into the evolution of NVIDIA's GPU architecture, from Ampere to Blackwell, explaining how each generation has improved computational performance through features like asynchronous data movement, tensor cores, and new memory architectures. The posts emphasize GPU programming paradigms and the potential cost savings from optimizing matmul performance. Future installments promise to explore hardware-specific optimizations and techniques to maximize performance, ultimately surpassing the current state-of-the-art provided by cuBLAS.

This past month has seen significant advancements and community engagement within the Modular ecosystem, highlighted by the release of the Modular Platform 25.5, which includes features such as Large Scale Batch Inference and new integrations to facilitate AI scaling. The platform is already being utilized in production deployments like SF Compute's Large Scale Inference Batch API, achieving cost reductions of up to 80% while supporting over 15 models. The Modular community has actively contributed to the ecosystem through various projects, including experiments with Gaussian splatting and probabilistic data structures, and has hosted meetups around the globe. Mojo, a key component of the Modular ecosystem, was featured in the 2025 Stack Overflow Developer Survey, marking its rapid adoption just two years after its launch. The community activities include notable contributions to GPU development, with ongoing challenges and educational content such as the GPU Puzzles series. The platform has also expanded its presence in the AWS Marketplace with applications like MAX High-Performance GenAI Serving Platform and MAX Code Repo Agent, both offering enhanced AI capabilities. Community contributions continue to enrich the ecosystem, with numerous developers having their work merged into the project, emphasizing the collaborative and innovative spirit driving the Modular platform forward.

Modular Platform 25.5 introduces several innovative features, including Large Scale Batch Inference, which is a highly asynchronous API developed in collaboration with SF Compute to efficiently manage AI workloads using Mammoth, a Kubernetes-native cluster orchestration layer. This release also launches the open-source MAX Graph API, allowing for the creation of GPU-accelerated graphs in Python, while enhancements to the API ensure model correctness through compile-time verification. The update further simplifies GPU development with new standalone Mojo Conda packages and lightweight MAX serving packages that significantly reduce deployment overheads. Seamless integration of MAX graphs into PyTorch workflows is enabled through custom operators, expanding the capabilities for extending PyTorch with MAX. With improved packaging and performance, Modular 25.5 offers a robust framework for AI developers seeking to optimize their systems and leverage GPU acceleration.