May 2026 Summaries

Lane Fiedler conducted an experiment to examine the impact of depth in Transformer models on reasoning tasks by pretraining two decoder-only Transformers from scratch, differing only in the number of attention layers (1 vs 12), on a dataset of 840 chain-of-thought conversations. The 12-layer model demonstrated a significantly lower training loss, indicating better fitting of reasoning patterns, while both models remained under 100 million parameters and were trained on Kaggle's T4 GPUs. Despite similar validation losses due to data scarcity, the experiment highlighted the importance of depth in capturing the reasoning distribution, supporting the idea that multiple layers facilitate iterative reasoning. The study also acknowledged limitations such as single seed testing and parameter differences between models, suggesting that further experiments could help disentangle depth from parameter count effects.

Profiling in PyTorch can be a daunting task due to its complexity and the dense traces it produces, but understanding how to navigate these traces is crucial for optimizing machine learning models. This introductory guide to using torch.profiler aims to demystify the process by starting with a fundamental operation—matrix multiplication followed by bias addition—and teaching how to interpret profiler outputs to drive optimization. The guide explains how to set up torch.profiler, read the profiler table and trace, and understand the chain of events from Python calls to CUDA kernel execution. It highlights common profiling challenges, such as overhead-bound algorithms and CPU-GPU offsets, and provides insights into operator fusion at the dispatcher level, as seen when using torch.compile. The guide emphasizes that while torch.compile offers potential performance enhancements, it also introduces additional CPU overheads that only amortize over larger workloads. By the end of this guide, readers will have a foundational understanding of how to use profiling tools in PyTorch to identify and address performance bottlenecks in their code, setting the stage for more advanced profiling techniques in subsequent parts of the series.

Dell Technologies World 2026 showcased the latest advancements in Dell Enterprise Hub (DEH), an on-premises platform for deploying open-source AI models on Dell infrastructure. The event introduced roughly twenty new model configurations optimized for Dell AI Servers and AI PCs, enabling rapid deployment from model selection to production. The release highlights include benchmarked deployment configurations, traceable container images, and new dell-ai SDK utilities for pre-deployment infrastructure verification. DEH now supports a range of platforms, including the NVIDIA B300 and GB10, with Goodput Scenarios for optimizing workload performance according to service level objectives (SLOs). The dell-ai SDK, an open-source tool co-developed by Dell and Hugging Face, facilitates scriptable model discovery and infrastructure verification, streamlining the deployment process. Dell Enterprise Hub continues to advance the adoption of open-source AI models, offering ready-to-use deployments across a broad spectrum of enterprise AI hardware.

The article explores the challenges and solutions associated with training large language models (LLMs) using reinforcement learning (RL), emphasizing the importance of maintaining the Token-In, Token-Out (TITO) invariant. It highlights the pitfalls of re-tokenizing model outputs, which can lead to unreliable gradient signals due to non-reversible tokenization processes. The recommended solution is to avoid re-encoding decoded tokens, using a buffer to keep track of the model's sampled tokens, thus maintaining structural integrity and preventing token drift. The article further discusses methods to ensure chat templates are prefix-preserving for tool messages, which is crucial for maintaining the consistency of the training loop. It contrasts two approaches: a lighter, more generic TITO loop and a heavier model-specific renderer, each with its advantages. The piece concludes by emphasizing the need to understand and verify the prefix-preservation property of chat templates for effective model training without re-implementing templating logic.

MiniMax's new architecture, M3, introduces a sparse attention mechanism that promises significant speed improvements, with 9.7× prefill and 15.6× decode speedup at 1 million tokens, as depicted in a diagram shared by R&D lead Skyler Miao. This advancement is part of a shift from the M2 model's full attention approach, which lacked the production readiness of M1's Lightning Attention. M3's design focuses on separating the tasks of selecting key-value (KV) pairs and computing attention, resulting in a streamlined process that employs block-level selection without compromising the expressive power of softmax attention. By adopting GQA over MLA and eliminating redundant branches, M3 achieves a balance between engineering efficiency and quality, aligning with the native sparse attention (NSA) principles. The design reflects a strategic choice to prioritize practical implementation speed and reusability of existing kernels over theoretical optimization, positioning MiniMax at the forefront of long-context open-source models as the industry standardizes around 1 million token contexts.

The blog post introduces "trimming," a technique for reducing the size of machine learning models by modifying or removing model weights, specifically focusing on vocabulary-related parts of the architecture. Unlike pruning, trimming targets the model's vocabulary size to optimize memory usage and computational efficiency without retraining, making it suitable for multilingual models. The discussion includes experiments on various models, demonstrating that trimming can maintain or even enhance performance while significantly reducing model size. The article explores the impact of trimming on different architectures, such as text embeddings, encoders, decoders, and vision-language models (VLM), and emphasizes the advantages of trimming over distillation and quantization. The post also touches on open questions related to the optimal number of tokens to retain, the order of trimming and fine-tuning, and its effect on biases, suggesting that trimming could offer a simple yet effective alternative for model optimization.

Jasper Research has released MONET, the largest open image-text dataset, designed to democratize access to high-quality data for training text-to-image models. Built from an initial pool of 2.9 billion images, it was refined to 104.9 million high-quality samples using a six-stage filtering process that includes aesthetic and safety pre-filtering, deduplication, and domain filtering, while ensuring a balanced distribution across diverse content categories. The dataset is paired with nano-t2i, a minimal codebase that allows researchers to train competitive diffusion models efficiently on a single GPU, significantly lowering the barriers to entry for developing high-quality text-to-image models. By providing free access to this meticulously curated dataset under an Apache 2.0 license, MONET addresses the reproducibility gap in AI research by enabling academic researchers and smaller companies to compete with closed-source commercial systems. With a mix of real and AI-generated images, MONET optimizes data quality without compromising model performance, as validated by its competitive results against larger commercial models on benchmarks like GenEval and DPG. While MONET mitigates challenges like geographic bias and caption inaccuracies, future improvements are anticipated to enhance multilingual capabilities and ensure broader cultural representation.

ITBench-AA is a strategic benchmark developed by Artificial Analysis and IBM to evaluate AI models on agentic enterprise IT tasks, specifically focusing initially on Site Reliability Engineering (SRE). The benchmark assesses models' abilities to diagnose complex Kubernetes systems by analyzing logs, traces, and infrastructure dependencies to identify root causes of incidents. Despite leveraging IBM's expertise in enterprise IT operations, all frontier models scored below 50% on these tasks, highlighting the challenge of this benchmark. The top-performing model, Claude Opus 4.7, achieved a 47% success rate, with other models like GPT-5.5 and Qwen3.7 Max closely following. The methodology involves using a Stirrup reference harness, a sandboxed environment where models can interact via shell commands, with performance scored based on precision at full recall. This setup ensures consistent comparison across models, with the benchmark revealing that models with longer investigation trajectories did not necessarily yield higher accuracy, and cost considerations also varied significantly among models.

Reachy Mini users can now operate their robots entirely locally without relying on external servers by deploying a local speech-to-speech backend. This setup includes a cascade approach using components like Silero VAD for voice activity detection, Parakeet-TDT for speech-to-text, llama.cpp for local language model processing, and Qwen3-TTS for text-to-speech. This local deployment ensures privacy, eliminates API costs, and allows users full control over the speech pipeline, enabling them to swap components as new models become available. The system's flexibility is enhanced by the ability to use various models for language processing, either running locally or via different external inference engines, which communicate through a standardized API protocol.

In a recent development, the process of asynchronous reinforcement learning (Async RL) has been made significantly more efficient by minimizing the data transfer between the trainer and inference engine. Traditionally, each training step required the entire model to be sent, which could be up to a terabyte for frontier models. However, it has been observed that between consecutive RL optimizer steps, over 98% of the weights remain unchanged, allowing for only the changed weights to be sent as a sparse safetensors file. This approach drastically reduces the payload size from gigabytes to mere megabytes. The implementation involves encoding the changes, uploading them to a Hugging Face Bucket, and fetching them with vLLM, which can operate independently on different servers or regions. This new method eliminates the need for shared clusters or complex networking setups, making Async RL more accessible and cost-effective while maintaining efficiency, especially for large-scale models.

A community discussion highlights frustrations with server capacity limitations affecting user access and functionality, with participants expressing their inability to initiate sessions or open new conversations for over a week. The article's author, specimba, reports ongoing issues, including authentication failures due to missing or invalid API keys for model providers like Anthropic, OpenAI, and HF Router. The community shares potential solutions such as setting API keys or using a .env file, while encouraging users to report persistent problems. Participants, including clem, lewtun, and akseljoonas, continue to face similar challenges and engage in dialogue to find resolutions.

Victor Mustar shares his experience of autonomously creating the LongCat-Video-Avatar 1.5 Space using ZeroGPU through a single agent session. This process was made possible with a Hugging Face PRO subscription, which offers 40 minutes per day of Blackwell GPU access and the ability to host up to 10 ZeroGPU Spaces. ZeroGPU is highlighted as a cost-effective solution, as it only engages the GPU when necessary, reducing idle costs. The agent uses tools like Codex CLI and Claude Code to iteratively design, deploy, test, and fix the application, streamlining the development process without human intervention. The approach leverages the Gradio SDK, PyTorch, and automated shell commands to optimize the model's performance and ensure successful deployment. This setup allows for scalable distribution and hosting of AI applications with minimal costs, making it accessible for developers to create and share viral AI apps efficiently.

In the rapidly evolving field of AI agents, terminology often outpaces a shared understanding, leading to confusion among both newcomers and seasoned practitioners. The article explores key terms such as "harness" and "scaffold" in the context of AI agents, emphasizing the importance of distinguishing between them for clarity in discussions. A harness is described as the execution layer that enables the model to act, while scaffolding provides the behavior-defining structure around the model. The glossary aims to offer practical mental models rather than enforce strict definitions, acknowledging the diverse usage of terms across different frameworks. It highlights the significance of context engineering, where designing what an agent perceives at each step can significantly influence its performance. The text underscores that while a model's policy defines its behavior, the agent is a comprehensive system that encompasses both the model and its operational framework, enabling it to interact with its environment effectively.

Borealis is an open-source, audio-language model developed by VikhrModels, designed to handle both Russian and English languages, and it aims to enhance audio understanding beyond transcription. The model utilizes Whisper3-large as the audio encoder and Qwen 4B as the language model backbone, with an adapter to bridge them. Borealis is developed to summarize lengthy recordings, answer content-related questions, and interpret tone and emotion. The training involved multiple datasets, focusing on the importance of native data over multilingual datasets and the nuanced role of text data in training. In terms of architecture, Borealis employs a frozen Whisper encoder, a four-times downsampler, and a fine-tuned Qwen3-4B language model using LoRA, resulting in a total of approximately 5 billion parameters. The model shows strong cross-lingual transfer capabilities, although native audio data still performs better, and excessive text data can degrade performance. Borealis addresses challenges such as noise and complex acoustic environments, highlighting the need for separate tuning for noisy audio. Additionally, Borealis offers practical insights into serving and integrating with transformer models, emphasizing the importance of pretraining and the challenges associated with audio longer than 30 seconds, heavy noise, and offline-only streaming.

In a novel experiment, the author explores the use of a genetic encoding system as an alternative to traditional system prompts for AI agents, encapsulating personality traits and behaviors in a compact genome string of around 120 characters. This genome encodes 64 trait loci with 251 distinct alleles and varying intensity levels, allowing the AI to exhibit diverse behaviors across different contexts, such as coding, medical, and legal domains. The experiment, conducted with minimal training on a small model, demonstrated a significant shift in AI outputs aligned with the genome's dictates, suggesting potential for more consistent and portable AI personas. The approach promises to drastically reduce the token cost of persona prompts, enhance output consistency, and even enable the breeding of AI agents by recombining genome strings, though it remains a speculative avenue for further exploration.

PapersWithCode has been relaunched by Niels Rogge from Hugging Face's open-source team, introducing new features to track state-of-the-art advancements across various AI domains. The updated platform now supports multiple metrics for benchmarks, such as Word Error Rate and Inverse Real-Time Factor for automatic speech recognition, and allows the submission of external papers from platforms beyond Arxiv, like GitHub and BiorXiv, which are automatically enriched with relevant tags and evaluations. The site also introduces paper lineage tracking, new popular methods, and the ability to screenshot leaderboards for easy sharing on social media. Additionally, the platform has expanded its evaluation database to include approximately 3,000 evals, with plans for further enhancements based on community feedback. Niels plans to facilitate communication through a new channel on the Hugging Face Discord server.

Nemotron-Labs Diffusion introduces a novel approach to language model generation through Diffusion Language Models (DLM), which generate multiple tokens in parallel and refine them iteratively, thus enhancing performance and allowing for token revision. This approach addresses the limitations of traditional autoregressive models, which generate text token-by-token and are constrained by memory and computational inefficiencies. The Nemotron-Labs Diffusion models, available in various scales and under the NVIDIA Open Model License, offer three generation modes—autoregressive, diffusion, and self-speculation—allowing developers to switch between them with minimal changes to their applications. This flexibility enables developers to achieve faster and more accurate text generation, while maintaining compatibility with existing workflows. Training these models involved pre-training on vast datasets and fine-tuning for enhanced performance, with support for deployment through SGLang ensuring broad usability.

An experiment explored whether compressed context states could effectively replace full attention mechanisms in language models, particularly in preserving weak, parallel instructions over long sequences. The experiment, conducted using a synthetic dataset, compared a traditional attention-based model with a model utilizing a compressed memory state across varying context lengths. Results indicated that the attention-based model outperformed the compressed model in both accuracy and speed, especially as context length increased. While the compressed model conceptually aimed to retain early rules without explicit classification, it failed to match the performance of attention, revealing that a naïve compression approach was insufficient. The findings underscore the robustness of full attention in handling tasks with complex rule retention requirements and highlight the need for more refined strategies in designing efficient context mechanisms. The study suggests future improvements could involve enhancing the preservation of weak constraints and optimizing implementation for parallel processing, rather than merely increasing compression.

In the realm of AI procurement, the traditional emphasis on using large-scale models is being challenged by findings that suggest specialization and alignment to specific tasks can yield superior performance, cost savings, and stability. Research from Dharma-AI highlights that a 3-billion-parameter specialized model outperformed larger commercial models in a specific OCR benchmark at significantly lower costs. This suggests that rather than focusing solely on parameter count, the training history and how closely a model's training has been aligned with its deployment task are critical variables influencing performance. The study indicates that specialization is not merely a compensatory approach for smaller models but a strategic measure of alignment that can yield better outcomes. This challenges enterprises to reconsider their AI evaluation frameworks to include distributional alignment as a key factor, potentially leading to the development of ecosystems of models tailored to specific domains and operational needs. The findings propose a shift in strategy, emphasizing the importance of model specialization and alignment over sheer scale.

Universities are increasingly incorporating AI into their curricula, but many focus on teaching students to use existing closed AI tools rather than building with open models, which limits their understanding of AI's inner workings. Open-source AI education is advocated as a sustainable approach that equips students to develop and customize AI systems, fostering innovation and adaptability in the workforce. Institutions like Stanford HAI, ETH Zurich, and EPFL, along with regulatory bodies such as the EU AI Act, emphasize the importance of open models for transparency and reproducibility. The shift toward open-source AI is supported by industry trends, with major companies restructuring their workforces to prioritize AI engineering skills. Hugging Face's Academia Hub is designed to help educational institutions navigate the open-source AI ecosystem by providing access to resources and infrastructure for hands-on learning and research. The push for open-source AI education aligns with the global demand for skilled AI developers, promising to better prepare students for evolving job markets and ensuring responsible AI development.

In a week-long journey across Beijing, Shanghai, and Hangzhou, Matt White, Global CTO of AI at the Linux Foundation, explored China's AI and robotics landscape, offering insights into the country's burgeoning technological ecosystem. During his visit, he engaged with AI labs, startups, and academic institutions, noting the distinct cultural and operational differences between Chinese and Western AI communities. White observed that Chinese AI labs prioritize open-source collaboration, architectural efficiency due to compute constraints, and a collective approach to innovation, contrasting with the more individualistic culture of Silicon Valley. The visit coincided with the release of several significant AI models and the introduction of new policy measures affecting AI development, highlighting China's strategic focus on building autonomous AI capabilities and advancing domestic chip production in response to export controls. White also emphasized the need for stronger global collaboration and open-source engagement to bridge the gap between the U.S. and Chinese AI ecosystems, advocating for open science as a means to foster shared progress despite geopolitical tensions.

LeRobot Humanoid is an open-source, low-cost humanoid robot project designed to advance robot learning by providing an accessible platform for experimentation. Priced around $2,500 depending on sourcing and shipping, the robot is constructed using 3D-printed parts, off-the-shelf components, and affordable electronics. Unlike many existing projects, LeRobot Humanoid offers a comprehensive package that includes hardware, assembly documentation, runtime tools, identification datasets, and training environments. This allows researchers and builders to design, build, modify, and test the robot in a closed-loop system from simulation to real-world deployment. The platform is particularly aimed at those interested in exploring the full robot-learning loop, from mechanical design and simulation to data collection and real-world control. While still experimental, the current release focuses on a bipedal platform with plans to enhance its robustness and expand capabilities, offering a unique opportunity for builders to iterate and contribute to open robot learning.

SANA-WM is a 2.6 billion parameter world model designed for image-to-video generation, initially developed for CUDA-based systems and later adapted to run on Apple Silicon, specifically on an M3 Max MacBook Pro with 128 GB unified memory. The adaptation required bypassing CUDA dependencies and implementing a memory-efficient, staged execution process due to Apple Silicon's shared memory constraints. This involved splitting the process into distinct stages—loading and unloading components sequentially—thus preventing simultaneous high memory usage. The runtime utilizes PyTorch MPS and Metal, avoiding CUDA-specific resources, and focuses on generating controlled video rollouts rather than a real-time interactive experience. While the port successfully enables local video generation on Apple hardware, future developments aim to create a more interactive, game-like experience by enhancing responsiveness and incorporating continuous state management and low latency updates. The runtime and patch set are publicly available, allowing others to replicate and further develop the implementation.

OlmoEarth v1.1 introduces a more efficient family of transformer-based models for processing remote sensing data, significantly reducing compute costs by up to three times compared to its predecessor, OlmoEarth v1, while maintaining similar performance levels. The updated model leverages a novel approach to tokenization by splitting Sentinel-2 satellite imagery into resolution-based patches, which helps in modeling important cross-band relationships without significant performance loss. This efficiency allows for more cost-effective and faster deployments of the model across national, continental, and global scales, making frequent updates of planet-scale maps more feasible for users. By training OlmoEarth v1.1 on the same dataset as its predecessor, the new release isolates the impact of methodological changes, thereby advancing the understanding of scientific principles in pretraining models for remote sensing. The model family is available in various sizes to accommodate different compute budgets, offering significant benefits for both developers and researchers in terms of cost savings and enhanced performance.

Tom Aarsen announced the release of the Ettin Reranker Family, a set of six new state-of-the-art Sentence Transformers CrossEncoder rerankers, each built on Ettin ModernBERT encoders. These models, ranging from 17 million to 1 billion parameters, are designed to enhance the accuracy of document retrieval systems by reordering results based on relevance scores. They employ a pointwise mean squared error (MSE) distillation from a strong teacher model, using a broad dataset of approximately 143 million query-document pairs. The rerankers are particularly efficient due to their architecture, which supports modern attention mechanisms like Flash Attention 2, offering significant speed improvements over previous models. The Ettin rerankers outperform existing models such as the MiniLM series on both MTEB and NanoBEIR benchmarks while maintaining high throughput. The release includes training recipes and data, making it accessible for further development and optimization by the community.

Codebases have historically been challenging to document, as much of the context behind decisions is often lost over time due to inadequate maintenance of comments, documentation, and logs. This issue is exacerbated by the rise of agents that produce context during code development, which is often stored in disorganized locations like personal laptops or Slack conversations. At Hugging Face, they propose using centralized storage buckets to store agent traces, which are detailed records of the decision-making processes that occur during coding. By synchronizing these traces into a single, accessible location, both human and agent reviewers can gain a comprehensive understanding of the changes made in a codebase. This approach aims to provide a durable home for agent-produced context, ensuring that future developers and agents are not starting from scratch when examining past work. Hugging Face utilizes their hf CLI tool to manage and sync these traces, enhancing the development experience by maintaining a clear history of code evolution.

PaddleOCR 3.5 integrates Optical Character Recognition (OCR) and document parsing tasks with the Hugging Face ecosystem by allowing PaddleOCR models to use Hugging Face Transformers as an inference backend. This new version offers a flexible inference-engine interface, enabling developers to select their preferred backend and configure specific options through the engine_config. It facilitates the integration of PaddleOCR's capabilities with Transformers-centered environments, making it advantageous for developers building applications like RAG, Document AI, and analytics, which rely on PyTorch and Transformers infrastructure. This release supports a smooth transition from document ingestion to downstream workflows, helping convert complex documents into structured data more efficiently. PaddleOCR 3.5 does not replace existing backends but provides additional flexibility, allowing developers to choose the best fit for their stack while maintaining PaddleOCR's management of the OCR and document parsing pipeline.

NVIDIA Cosmos Predict 2.5 is a world model designed for generating realistic videos based on text, images, or video prompts and can be fine-tuned to specific domains like robot manipulation. Fine-tuning large models is often resource-intensive, so techniques like LoRA and DoRA are used to inject smaller, trainable adapter modules into a frozen base model, making the process more efficient and flexible. By utilizing these methods, the model can be fine-tuned on a single GPU while maintaining general knowledge. This process allows for the generation of synthetic robot trajectories, which are useful for training robot policies without the high cost of collecting real-world data. The guide details the parameter-efficient fine-tuning process using the diffusers and accelerate libraries, implementing LoRA and DoRA, and evaluating the model's performance based on physical plausibility and instruction-following metrics. The study concludes that fine-tuning for 100 epochs on 8 H100 GPUs significantly improves video generation quality in terms of temporal stability, geometric consistency, and task completion, with LoRA and DoRA offering different advantages based on memory and stability requirements.

The Open Agent Leaderboard is a newly launched evaluation framework designed to assess the performance of general-purpose AI agents by considering the entire system, rather than just the models. It evaluates agents across six diverse benchmarks, including tasks in coding, customer service, and personal assistance, to measure how well they adapt to various settings without specific tuning. The initiative emphasizes the importance of agent architecture, revealing that while model choice remains a key factor, the design of the agent system significantly impacts performance and cost-effectiveness. The leaderboard aims to provide a transparent, community-driven platform for evaluating and improving AI agents, encouraging contributions from developers, benchmark creators, and researchers to expand its scope and utility. The project underscores the need for open evaluation and collaboration to advance the development of general-purpose agents capable of handling a wide range of tasks efficiently.

VIDRAFT's Darwin Family introduces an innovative approach to developing frontier-level reasoning language models (LLMs) without relying on traditional gradient-based training methods. Instead, the Darwin Family recombines the weight spaces of existing model checkpoints using a 14-dimensional adaptive genome, MRI-Trust Fusion, and an Architecture Mapper, which allows for the integration of different architectural elements. This method has led to the creation of Darwin-28B-Opus, a model achieving an 88.89% score on the challenging GPQA Diamond benchmark without any gradient-based training steps. The approach significantly reduces the computational cost typically associated with training high-capability models and demonstrates that open-source LLMs contain latent capabilities that can be unlocked through recombination. The framework's success suggests a shift in focus from traditional training to the extraction and recombination of existing model capabilities, potentially lowering the barriers to producing state-of-the-art reasoning models.

Vividh-ASR addresses studio-bias in existing ASR models for Indic languages by introducing a benchmark stratified by acoustic complexity and a Whisper fine-tuning recipe that enhances model robustness across various conditions. The key discovery is that fine-tuning Whisper with a high learning rate significantly outperforms existing models, especially for Hindi and Malayalam, without needing architectural changes or proprietary data. The study reveals that training on harder conditions first benefits the performance for Malayalam, while for Hindi, the high learning rate alone suffices. This advancement enables a 244M parameter Whisper model to surpass larger models on overall Word Error Rate (WER). Adalat AI focuses on developing robust ASR models to serve the Indian judiciary, tackling challenges related to spontaneous and varied-condition audio as well as operational efficiency for large-scale concurrent use. The research challenges standard fine-tuning assumptions, suggesting that a high learning rate and reverse curriculum learning (hard-to-easy) are more beneficial for certain languages, although the curriculum order's impact varies between Hindi and Malayalam. The release includes models and benchmarks that significantly outperform existing baselines, providing new insights for practitioners working on low-resource Indic languages.

In this article, the authors explore how asynchronous batching can significantly enhance GPU utilization and performance during inference by allowing CPU and GPU tasks to run concurrently. Traditional synchronous batching results in inefficiencies as the CPU and GPU take turns, leading to idle periods that contribute to nearly a quarter of total runtime. By implementing asynchronous batching with CUDA streams and events, the CPU can prepare the next batch while the GPU processes the current one, minimizing idle time. This approach involves using separate streams for different GPU operations and ensuring synchronization with events to prevent data corruption and ensure data is ready when needed. The method successfully increases GPU active time from 76% to 99.4%, resulting in a 22% speedup in total generation time without requiring new kernels or model changes. The implementation is part of the transformers library, and future articles will explore additional optimizations for further performance improvements.

The guide provides an in-depth examination of how companies can use Hugging Face to comply with SOC 2 and ISO 27001, focusing on AI model supply chain governance. It highlights the increasing scrutiny on AI models and datasets by auditors who now extend compliance frameworks traditionally applied to code dependencies and SaaS to AI systems. Hugging Face is SOC 2 Type II certified and GDPR compliant, with additional compliance features available through various plan tiers—Free, Team, Enterprise, and Enterprise Plus. These tiers offer different levels of governance capabilities like audit logs, SSO, and user download analytics, which are crucial for satisfying auditor requirements. The guide underscores that while Hugging Face provides foundational features for AI model documentation and integrity, such as model cards and DOI assignments, the higher-tier plans facilitate comprehensive control and governance over AI model usage, ensuring alignment with emerging regulatory standards and auditor expectations.

The Granite Embedding Multilingual R2 release introduces two new Apache 2.0 licensed multilingual embedding models, offering significant advancements in retrieval quality across 200+ languages and code. The compact 97M-parameter model, built on ModernBERT architecture, achieves outstanding retrieval scores, outperforming other sub-100M parameter models on the MTEB Multilingual Retrieval benchmark, while the full-size 311M-parameter model ranks second among open models under 500M parameters. These models, which feature 32K-token context handling and cross-lingual code retrieval, are designed to be enterprise-ready, having been trained with IBM-curated datasets and stringent governance processes to ensure commercial use safety. They are compatible with sentence-transformers and other frameworks like LangChain, LlamaIndex, Haystack, and Milvus, requiring no task-specific instructions for integration. The models also support Matryoshka Representation Learning, allowing flexible embedding dimensions to optimize storage and computation costs. These innovations represent a leap forward in multilingual model efficiency, offering robust performance in cross-lingual, code, and long-document retrieval tasks.

In May 2026, Google Translate added Sango, the national language of the Central African Republic, to its supported languages, marking a significant step for zero-resource African languages in AI. The addition highlights the need for domain-specific vocabulary and grammar infrastructures, which general-purpose translations often lack. To address this, SangoAI was developed using a method called vocabulary-augmented prompting, which involves using a curated lexicon and language-specific prompts with a general-purpose language model, avoiding the need for large parallel corpora or fine-tuning. This approach, although not as theoretically elegant as classical neural machine translation methods, provides production-quality translations for languages like Sango and can be adapted to other low-resource African languages. The project emphasizes the importance of specialized vocabulary and grammar infrastructure for effective communication in various domains such as healthcare and education. The success of SangoAI demonstrates a scalable solution for the approximately 2,000 African languages that need similar support, with future plans to expand to other languages like Ewondo and Lingala.

In 2026, running large language models (LLMs) locally has transitioned from a niche experiment to a viable production option due to advances in open-source and open-weight models. This shift offers more control over data, as models can be deployed on personal devices, private clouds, or on-premise infrastructure without relying on third-party APIs, thus addressing concerns about data residency and privacy. The article outlines various LLMs suited for different hardware configurations and use cases, highlighting models like Qwen3 for general use, Devstral for coding, and Llama 4 Scout for long-context applications. It emphasizes the importance of understanding licensing types, such as Apache 2.0 and MIT, to ensure compliance with commercial use, and discusses the strategic advantages of local deployment, including cost efficiency at scale and enhanced trust in data handling. The text advises starting with smaller models that match the available hardware capabilities to avoid performance issues, and it also suggests tools like Ollama and vLLM for local model deployment and management.

Advancements in AI are pushing beyond simple tasks towards more complex, long-term missions that require persistence and strategic thinking, exemplified by AI agents outperforming human hackers in discovering cybersecurity vulnerabilities. This evolution is giving rise to "None Person Companies" where AI autonomously manages operations in sectors like e-commerce and finance. The realization of this future hinges on enhancing AI with memory, continual learning, and self-judging capabilities, aiming for self-evolving systems that can independently improve through self-training. This trajectory towards Artificial General Intelligence (AGI) implies a transformative shift in technology, challenging existing architectures and demanding ethical and regulatory considerations as AI begins to reshape industries and potentially surpass human capabilities.

Foundation model training and inference on AWS involve a complex interplay of advanced infrastructure, resource orchestration, software stacks, and observability tools. The article explores the evolution from a singular scaling approach to three complementary regimes—pre-training, post-training, and inference—emphasizing the need for tightly coupled accelerator compute, high-bandwidth low-latency networking, and scalable distributed storage. It delves into how AWS infrastructure, including multi-node accelerators and EFA networking, supports these processes, with orchestration handled by systems like Slurm and Kubernetes. The ML software stack, incorporating PyTorch and specialized libraries, enhances distributed training and inference capabilities, while observability through Prometheus and Grafana ensures efficient operation and troubleshooting. The interconnected layers, from hardware to software, highlight the importance of precise configuration to avoid performance bottlenecks and optimize the foundation model lifecycle on AWS.

Between May 2024 and May 2026, significant advancements in open-weight AI models outpaced Moore's Law on MacBook Pro laptops with static hardware specifications. Despite the maximum memory remaining at 128 GB across three generations of MacBook Pro chips, the Artificial Analysis Intelligence Index score of the most sophisticated model available for these laptops jumped from 10 to 47, demonstrating more than twice the improvement rate of traditional hardware developments as predicted by Moore's Law. This remarkable progress was driven by innovations in software, specifically through the use of sparse Mixture of Experts (MoE) models, aggressive quantization techniques, and reasoning-tuned dense models. These advancements allowed for more efficient memory usage and greater intelligence per parameter, enabling the execution of increasingly complex AI models on unchanged hardware. As the hardware ceiling remains a potential constraint, any future improvements will likely rely on further model optimizations unless hardware upgrades occur.

Safety evaluations of AI models should consider the potential impact of test-time compute, as a model that seems safe under limited evaluation conditions may become unsafe when adversaries apply larger, adaptive, and economically rational computational resources. The conventional approach of assessing whether a model can perform dangerous actions is inadequate for modern AI systems, where adversaries can employ extensive inference-time efforts like generating numerous prompt variants, using other models to improve attacks, or employing adaptive compute allocation. This shift emphasizes the need for evaluations that factor in the broader risk surface, which includes the model's behavior under varying budgets, attacker strategies, and deployment configurations. The economic rationale for adversaries to invest significant resources in attacks further complicates this landscape, as the potential payoff can justify high expenditure. Static safety checks remain useful but are insufficient for systems capable of longer reasoning, adaptive search, and tool use. As a result, safety evaluations should incorporate test-time compute into the threat model, providing risk assessments that account for different levels of adversarial effort and labeling safety claims with the applicable conditions.

In the AI industry, there is a growing critique over the use of unlicensed data and the hidden exploitation of labor in the development of AI models. The industry relies on two types of human work: existing content such as novels and scientific papers, often scraped without permission for training base models, and newly created tasks like annotation and moderation to refine these models for commercial use. This practice raises ethical concerns as companies like Meta, Anthropic, and OpenAI face legal challenges for copyright infringements, with court cases revealing the unauthorized use of copyrighted material. Additionally, the labor involved in refining AI models is often underpaid and involves exposure to harmful content, with companies distancing themselves from the workers by using contractors. There is a push for more ethical AI development practices, emphasizing the use of licensed data, fair labor practices, and transparency in data usage. Alternative datasets like The Common Pile demonstrate the potential for developing competitive models without unlicensed data, challenging the industry's claim that using web-scale unlicensed text is necessary. Urro aims to build AI models with a clean provenance, focusing on ethical data sourcing and labor practices, highlighting the need for operational standards that prioritize ethical considerations over cost-cutting measures.

OncoAgent is an open-source, privacy-preserving clinical decision support system tailored for oncology, leveraging a dual-tier multi-agent framework to enhance clinical decision-making while maintaining patient data sovereignty. It features a dual-tier architecture, using a speed-optimized 9B parameter model for simpler queries and a deep-reasoning 27B parameter model for complex cases, both trained on a comprehensive dataset of real and synthetic oncological cases. The system employs a Corrective Retrieval Augmented Generation (RAG) pipeline to ensure recommendations are grounded in physician-grade guidelines and incorporates multiple safety layers, including a reflexion safety loop and human-in-the-loop (HITL) validation, to prevent hallucinations and ensure reliability. Hosted on AMD Instinct MI300X hardware, OncoAgent eliminates the need for proprietary cloud services, thus adhering to stringent privacy regulations like HIPAA and GDPR. This system represents a significant advancement in deploying scalable, reliable, and privacy-compliant AI solutions in clinical environments, demonstrating that state-of-the-art clinical AI can be achieved without reliance on proprietary infrastructure.

MedQA is an innovative project that fine-tunes the Qwen3-1.7B language model for medical question answering using AMD hardware and ROCm, highlighting a significant departure from the conventional reliance on NVIDIA's CUDA. The model, trained on the MedMCQA dataset, uniquely provides both the correct answer and an explanation for its choices, enhancing its clinical utility. The project showcases the feasibility of using AMD's Instinct MI300X, with its substantial 192 GB HBM3 memory, to train large models without resorting to quantization, thereby ensuring cleaner training outputs. Utilizing LoRA for fine-tuning, MedQA manages to effectively train only a small fraction of the model's parameters, significantly reducing memory usage and training time. The project further demonstrates the compatibility of the HuggingFace ecosystem with ROCm, proving that the same training code can be effortlessly adapted from CUDA to ROCm with minor environment adjustments. As a result, MedQA not only establishes the practical applicability of AMD hardware for complex AI tasks but also emphasizes the importance of explanatory outputs in medical AI, setting a precedent for future developments in this field.

EMO is a newly released mixture-of-experts (MoE) model designed to foster emergent modularity without relying on human-defined priors, enabling efficient use of resources by activating only a small portion of its experts for specific tasks. Unlike traditional large language models, which operate as monolithic systems, EMO allows for the selective use of expert subsets, maintaining near full-model performance even when only 12.5% of its experts are engaged. This model aims to overcome the limitations of standard MoEs, which often specialize in low-level lexical patterns, by encouraging experts to form coherent groups that align with semantic domains. During pretraining, EMO uses document boundaries as a supervisory signal to ensure tokens from the same document activate similar experts, promoting domain specialization. The model's effectiveness is demonstrated through its ability to maintain performance on general-purpose benchmarks, even with reduced expert subsets, and its modular design supports flexible deployment with improved memory-accuracy trade-offs. EMO's architecture and training approach provide a foundation for developing modular language models that are easier to deploy, adapt, and interpret, facilitating further research into expert selection and composition.

CyberSecQwen-4B is a specialized defensive cybersecurity model developed for tasks such as CWE classification and CTI Q&A, designed to operate locally on a 12 GB consumer GPU. It aims to address the limitations of large frontier models, such as high API costs and privacy concerns, by offering a smaller, specialized solution that retains a significant portion of the accuracy of larger models while being deployable in sensitive environments. Trained on an AMD Instinct MI300X, the model demonstrates competitive performance against larger counterparts, like Cisco's Foundation-Sec-Instruct-8B, by achieving notable accuracy in specific cybersecurity tasks. The training process utilized Apache-2.0-clean data and focused on maintaining the integrity of classification tasks without compromising on performance. This initiative highlights the importance of local, specialized models in cybersecurity to keep sensitive data in-house and respond effectively to automated adversarial threats, while offering flexibility for deployment in various environments.

The article provides a comprehensive guide for IT, security, and ML platform teams on integrating Hugging Face models with JFrog Artifactory, especially focusing on the changes required by June 2026. It explains the transition from legacy repository layouts to a new Machine Learning repository format, emphasizing the need for enterprises to adapt due to evolving Hugging Face platform dynamics and increasing AI adoption. Key challenges such as HTTP 429 rate limits in proxy environments and the limitations of Artifactory’s Xet protocol support are highlighted, alongside solutions like upgrading to Hugging Face Enterprise Plus for better rate limits and governance. The guide also discusses the benefits of Hugging Face’s Model Gateway for managing internal model registries with true content-addressed storage, overcoming limitations associated with Artifactory’s current integration. It suggests that while Artifactory is effective for universal artifact management, combining it with Hugging Face Enterprise Plus ensures a more robust architecture that addresses higher rate limits, organizational identity, and audit trails.

QVAC MedPsy represents a significant advancement in medical and healthcare language models, specifically designed for deployment on edge devices like smartphones and wearables. Developed by Tether Data's AI Research group, these models, with 1.7B and 4B parameters, offer medical reasoning capabilities that rival models several times their size, setting a new standard for efficient medical AI. The MedPsy-1.7B model outperforms larger models such as Google's MedGemma-1.5-4B-it, while the MedPsy-4B model exceeds the performance of the MedGemma-27B-text-it model, despite being significantly smaller. These models achieve high parameter and token efficiency, reducing latency and computational costs, thereby enabling clinical-grade AI in resource-constrained settings. The models are designed for private, on-device inference, preserving patient privacy and data security, and are made available under the Apache 2.0 license for research and educational purposes. The comprehensive evaluation across various medical benchmarks demonstrates the models' capability in accurate, real-time clinical decision support, marking a shift towards more accessible and secure medical AI applications.

The study explores the use of video compression as a training strategy to enhance the robustness of machine learning models for autonomous vehicle (AV) systems, focusing on depth estimation tasks sensitive to video transformations. The researchers propose treating compression as data augmentation, allowing models to learn geometric representations that withstand telematics challenges. By fine-tuning models with compressed video inputs, they achieved significant reductions in validation errors, improving model stability and geometric restoration without compromising the performance on uncompressed inputs. The approach demonstrated a notable 35.2% reduction in video size while maintaining high-fidelity depth predictions, proving that compression artifacts can be effectively neutralized, thus ensuring safe and accurate perception in AV pipelines. This methodology offers a cost-effective solution, allowing AV systems to scale efficiently by integrating compression augmentation into model training, enhancing resilience to telematics bottlenecks like video compression.

The Open ASR Leaderboard, launched in September 2023, has become a significant platform for benchmarking speech recognition models, attracting over 710,000 visits. The platform aims to enhance transparency and standardization in evaluating ASR models by integrating public datasets and recently curated high-quality private datasets from Appen Inc. and DataoceanAI, which cover diverse accents and speech styles. These private datasets are kept confidential to prevent "benchmaxxing," where models might artificially boost performance without real-world robustness. The leaderboard maintains openness by open-sourcing its UI code and evaluation scripts, which allows for community contributions and improvements. Standardization is achieved by normalizing model outputs and transcripts, ensuring consistency in punctuation and casing. The leaderboard's objectives include capturing the nuanced performance differences across models tailored for various accents, languages, and use cases, and it provides options to customize evaluations with private and public datasets to reflect specific application needs. The platform continues to evolve to include evaluations under noisy real-world conditions and is open to feedback and contributions to further refine its benchmarking capabilities.

Shane Crownelius shares his journey from an AI enthusiast to an innovator in the field, highlighting his creation of compact AI models like FANT and Sparrow. Initially skeptical about the practical application of AI empathy scores, Shane found unexpected success with a fine-tuned model that excelled in emotional intelligence, leading to a lucrative ghostwriting job. His focus shifted to developing small, efficient models, resulting in the FANT series and Sparrow, a math model that outperformed much larger models on complex mathematical tasks. Despite the challenges of integrating Sparrow's methods into FANT's architecture, Shane remains committed to refining his models and openly shares his progress and challenges with the community, inviting collaboration and discussion on platforms like GitHub and Discord.

The agentic robotics app store for Reachy Mini, an open-source desktop robot, allows users to create apps without needing a robotics background by leveraging AI agents to write and deploy code based on plain English descriptions. This initiative has led to the creation of over 200 apps by more than 150 creators, many new to robotics, demonstrating the accessibility of robotics development. The app store is entirely open-source, hosted on the Hugging Face Hub, and encourages a collaborative environment where apps can be forked, modified, and shared, much like the iPhone App Store revolutionized app development for mobile devices. A notable example is Joel Cohen, a 78-year-old CEO, who developed a voice-controlled AI co-facilitator for Zoom sessions using Reachy Mini, emphasizing the platform's accessibility and versatility. The open-source nature of the hardware and software promotes innovation and democratizes robotics, enabling a diverse range of users to build and share their creations easily.

The transition from vLLM V0 to V1 in PipelineRL focused on ensuring inference correctness before applying any reinforcement learning (RL) objective corrections. This migration aimed to eliminate discrepancies in how token log probabilities were computed, which are crucial for training dynamics. Key issues addressed included logprob semantics, runtime defaults, inflight weight updates, and the precision of the final projection, with each fix aiming to align V1's behavior with the V0 reference. The backend corrections ensured the V1 engine returned logprobs and runtime behavior that matched trainer expectations, preventing the need for premature objective-side corrections that could obscure training outcomes. The successful migration underscored the importance of verifying backend correctness before implementing additional RL objective improvements.

This tutorial provides a step-by-step guide for building a Legal Retrieval Augmented Generation (RAG) application using Python, aimed at beginners familiar with Python and Large Language Models (LLMs). The process involves using tools such as semchunk for semantic chunking, Kanon 2 Embedder and Reranker for embedding and reranking tasks, LangChain for the RAG framework, and Gemini for generative tasks. The goal is to address the limitation of LLMs in accessing updated information by retrieving relevant context to feed into an LLM, thus reducing hallucinations. The tutorial walks through various stages including dataset preparation, semantic splitting, embedding, storing vectors, retrieval, reranking, and generating answers with the context provided. The example uses Australian legal cases to demonstrate how updated and relevant information can be efficiently retrieved and utilized to answer legal queries effectively, emphasizing the importance of an updated information source and efficient retrieval to enhance the accuracy and reliability of LLM outputs.

A 20-year-old developer embarked on a solo project to build a French language model from scratch using a GTX 1080 Ti, aiming to understand every step of the process rather than simply fine-tuning an existing model. This involved creating a custom pipeline that included data collection, cleaning, tokenization, and training, with a model architecture inspired by LLaMA rather than traditional GPT-2. The dataset was derived from an AI-rewritten version of French Wikipedia to ensure uniformity in style, resulting in a 15-million-parameter model optimized for French. Training was structured over three phases—denoising, curriculum learning, and contrastive learning—though a power outage interrupted progress at the 10th epoch. Despite technical challenges and a limited computing setup, the project demonstrated the feasibility of developing a language model independently, with plans to expand the dataset and continue training on more robust cloud infrastructure to enhance the model's understanding and application across different domains.

A multilingual benchmark called "Talking to a 4-Year-Old" has been developed to evaluate AI companions for children, comprising 2,312 conversational prompts in 23 languages and assessed using four language models. The initiative arose from real incidents involving voice assistants providing unsafe guidance to children, highlighting the need for child-appropriate AI evaluation criteria. Unlike existing benchmarks, which cater to adults, this project focuses on children's interactions and safety, using real conversations from apps like Octo Kids as a foundation. The benchmark categorizes prompts into eight areas, including safety redirection and emotional support, and is assessed using a rigorous rubric system. Evaluations were carried out by multiple language models, and the responses were judged by five independent judges to ensure reliability. The entire dataset, alongside model responses and judge scores, is open source, aiming to enhance the development of safer AI systems for children.