November 2025 Summaries

In this article, Ulrick BLE discusses the development and evaluation of multimodal rerankers, specifically focusing on a reranker based on Qwen 3 VL 2B that surpasses the Jina Reranker M0 in performance, inference speed, and model size. The author explores the evolving field of multimodal retrieval and the crucial role of rerankers in enhancing the relevance of retrieved documents, particularly in visually rich corporate scenarios where traditional text-based methods fall short. By creating a benchmark dataset for evaluating multimodal rerankers and experimenting with reinforcement learning (RL) strategies, Ulrick BLE highlights the challenges and opportunities in optimizing rerankers for multimodal contexts. The article also delves into technical aspects such as attention implementation and inference optimization, emphasizing the potential of using pretrained language model (LM) heads for efficient performance. Despite achieving competitive results, the attempt to apply RL for reranking did not yield conclusive outcomes, indicating the complexities involved in multimodal environments.

CaptionQA is a benchmark developed to address the gap between academia and industry in the evaluation of captions, emphasizing that captions serve as crucial infrastructure for various industrial applications beyond merely describing images. While academic approaches often treat captioning as a descriptive task, industry requires captions to function as information interfaces that support tasks like search, recommendation, document structuring, and agent reasoning. The CaptionQA framework evaluates captions using a simple, scalable question-answering approach that emphasizes accuracy and task effectiveness, rather than traditional descriptive metrics. It is designed to be adaptable, allowing for domain-specific benchmarks, and highlights the importance of re-prioritizing captioning as an independent task in the development of multimodal systems. This shift aims to enhance the expressive capabilities of models, ensuring that captions accurately and efficiently convey task-relevant information.

MCP Blockly is an innovative educational tool developed to bridge the gap between beginner-friendly programming environments like Scratch and the complex realm of professional AI development. Created for the MCP's 1st Birthday Hackathon, it offers a visual interface for building Model Context Protocol (MCP) servers using block-based logic, supplemented by an AI assistant that acts as a collaborator rather than a mere tool. This assistant employs a custom Domain Specific Language (DSL) to interpret and manipulate the visual workspace, enabling it to perform complex tasks and generate Python code for deployment on platforms like Hugging Face. By fostering an interactive, hands-on learning environment, MCP Blockly aims to develop genuine understanding and intuition in students, encouraging them to actively engage with AI development rather than passively relying on automated solutions.

Gemini-3 is a leading AI model that excels in various legal benchmarks, demonstrating strong factual recall and legal reasoning across multiple jurisdictions, but it exhibits a tendency towards confident hallucinations when uncertain. The model performed well in assessments like LegalBench and Indian Law QA, showcasing near-perfect accuracy in statutory recall and binary legal judgments, but it struggled with tasks requiring intricate reasoning, such as the GreekBarBench and Wilful Misconduct tests. Despite its impressive ability to apply legal reasoning and handle complex legal queries, Gemini-3's overconfidence and occasional misalignment with binding precedents indicate the necessity for human oversight in its applications. Its performance highlights the potential of AI in legal contexts, yet underscores the critical importance of ensuring accuracy and reliability to avoid misleading outputs. Aggregate feedback rates the model's capability highly, suggesting it is on the cusp of bridging the gap with human legal practitioners, albeit with a cautionary note on its reliability.

Jobly is an AI-powered gig marketplace developed to enhance job matching in the gig economy by employing semantic search, vector embeddings, and Retrieval-Augmented Generation (RAG) instead of traditional keyword matching. This innovative approach was designed during the Hugging Face Winter Hackathon 2025 to address the limitations of keyword-based systems, which often miss synonyms and context, leading to mismatches. Jobly's three-tier matching architecture begins with TF-IDF for basic matches, progresses to vector embeddings for semantic understanding, and culminates with RAG to incorporate metadata and optimize results. This system uses models like LlamaIndex and HuggingFace to create a robust platform that understands the meaning of job descriptions and worker profiles, providing a more precise and context-aware matching process. Through this integration, Jobly achieves high precision and explainability without relying on expensive infrastructure, offering a scalable solution for matching gig workers with suitable opportunities.

Researchers Zhanhui Zhou and Lingjie Chen have developed a method to transform a standard BERT model into a conversational chatbot using a minimal amount of open-source instruction-following data and the diffusion framework, showcasing that such adaptation requires only supervised finetuning rather than extensive generative pretraining. They introduce dLLM, an open-source framework that standardizes the training, inference, and evaluation of diffusion language models (DLMs), addressing current barriers such as the lack of a unified framework and high computational costs. The framework supports easy reproduction of experiments and includes open implementations of previously unavailable algorithms. Their ModernBERT-Chat model, finetuned on instruction-response pairs, demonstrates performance close to the Qwen1.5-0.5B benchmark across several tests, suggesting that the original masked language modeling pretraining of BERT already imparts sufficient knowledge for diffusion-based generation. The researchers encourage further community contributions to enhance dLLM's capabilities, aiming to make it a comprehensive platform for DLM research.

The introduction of direct dataset editing on the Hub marks a significant shift in dataset workflows for AI, eliminating the need for the traditional download, edit, and upload cycles. This new feature allows collaborative dataset curation, enabling multiple users within an organization to make commits, review changes, and enhance data quality with full versioning and traceability. Currently, editing is possible for datasets containing a single CSV file with textual columns where users have write access. The process involves inspecting the dataset for errors, toggling an edit mode to correct issues, and committing changes with descriptive messages, which are then versioned for easy traceability. The platform is expected to evolve, with future enhancements likely to include AI models that expedite and improve data curation directly in the browser.

StepWise Math is an AI-powered application developed for the Model Context Protocol (MCP) 1st Birthday Hackathon, designed to transform static mathematical problems into interactive, visual proofs. The application leverages Gradio and the Gemini models to create a dynamic learning experience, allowing users to visualize and interact with mathematical concepts such as triangle angles and probability distributions in real-time. With a two-stage AI pipeline—Gemini 2.5 Flash for analyzing input and Gemini 3.0 Pro for building applications—the tool acts as a "Digital Montessori" for middle school students, breaking down complex proofs into accessible steps on an HTML5 canvas. The project also integrates with various MCP clients, enabling seamless interaction with AI agents, and emphasizes Spec-Driven Development and Vibe Coding to streamline the creation process. Future plans for StepWise Math include developing a persistent library for users and a teacher mode for classroom applications, showcasing the potential for AI to enhance educational engagement and understanding in mathematics.

Continuous batching is an optimization technique aimed at enhancing the throughput of large language models (LLMs) by processing multiple conversations in parallel without unnecessary computational overhead. This approach builds on several key components: attention mechanisms, KV caching, chunked prefill, ragged batching, and dynamic scheduling. The attention mechanism allows for efficient token interaction, while KV caching reduces computation by storing previously calculated states for reuse. Chunked prefill manages large initial prompts by splitting them into smaller, manageable chunks to fit within memory constraints. Ragged batching eliminates padding waste by concatenating prompts and using attention masks to control token interaction, thus maximizing memory usage. Dynamic scheduling further optimizes throughput by swapping completed prompts with new ones, ensuring continuous and efficient resource utilization. These techniques collectively enable modern LLMs to serve multiple users concurrently, exemplified by services like ChatGPT.

FLUX.2, developed by Black Forest Labs, is an advanced image generation model that builds upon its predecessor, Flux.1, with a new architecture designed from the ground up. Unlike Flux.1, FLUX.2 employs a single text encoder, Mistral Small 3.1, simplifying prompt embedding by stacking outputs from intermediate layers. The model retains the multimodal diffusion transformer (MM-DiT) + parallel DiT architecture but introduces modifications such as shared modulation parameters across transformer blocks and the elimination of bias parameters, leading to a more streamlined structure. FLUX.2 supports both image-guided and text-guided generation, allows multiple reference images, and provides advanced prompting techniques. It requires significant memory resources, with various strategies available to optimize inference and fine-tuning on consumer-grade hardware. Users can leverage techniques like LoRA fine-tuning, remote text encoding, and quantization to maximize efficiency and performance, making FLUX.2 a versatile tool for complex image generation tasks.

Hugging Face's Daily Papers page serves as a dynamic platform for AI researchers and developers to engage with current advancements in the field by featuring over 10,000 high-quality research papers curated by the community. Users can submit papers, either individually or as part of an organization, and claim authorship to enhance personal visibility and contribute to a growing repository of AI knowledge. The platform encourages interaction through comments and discussions, offering real-time feedback and fostering collaboration across the global AI community. Additional features include multilingual comments with translation support, integration with arXiv for seamless access to related models and datasets, and a voting system to highlight impactful research. Users can subscribe to receive daily updates, ensuring they stay informed about trending research and new developments.

Research agents are emerging as a crucial application of AI, capable of processing vast amounts of information and synthesizing insights instantly, thus overcoming human limitations such as memory and reading speed. The development of state-of-the-art research agents involves creating sophisticated software layers, known as agent harnesses, to manage context, orchestrate tasks, and handle errors in rapidly evolving AI models. The authors highlight the importance of designing these systems to adapt to future model improvements without relying on fixed assumptions, as well as the necessity of context engineering to maintain efficient, relevant data retrieval. By leveraging advanced search features and focusing on streamlined toolsets, they reduced token consumption and achieved state-of-the-art results while maintaining reliability and efficiency. Through a combination of simplified orchestration and careful monitoring, the authors advocate for a balance between autonomy and control in building production-grade agents, emphasizing that qualitative improvements in reliability and efficiency are more valuable than optimizing for numerical evaluation scores.

OVHcloud has become an Inference Provider on the Hugging Face Hub, enhancing the platform's capabilities for serverless inference and allowing users to easily access and utilize popular open-weight models like gpt-oss and Llama directly from Hugging Face. The integration is seamless with Hugging Face's client SDKs for both JavaScript and Python, facilitating the use of various models through preferred providers. OVHcloud AI Endpoints offer a fully managed, serverless service with competitive pricing and are designed for production use, delivering fast response times suitable for interactive applications. The service is hosted in European data centers, ensuring data sovereignty and low latency for European users, and supports advanced features for text and image processing. Users can set their API keys for different providers in their account settings, and billing is handled according to whether requests are made directly through providers or routed via Hugging Face. Hugging Face's PRO plan offers additional benefits, including Inference credits and higher usage limits, encouraging users to provide feedback through its community platform.

AGEFI and Dragon LLM have launched the LLM Open Finance Initiative, introducing open-source language models tailored for financial applications. The initial release includes two 8-billion-parameter models, optimized for tasks such as financial reporting analysis, risk assessment, and financial translation, with an emphasis on English and French languages. Developed as part of the France 2030 initiative and supported by Bpifrance, these models demonstrate superior performance on financial tasks compared to base models and other high-performing finance models. They maintain multilingual capabilities and general domain knowledge, making them versatile for various applications. The initiative aims to democratize access to advanced financial AI models, encouraging innovation in financial technology and economic research. For teams requiring more robust capabilities, the LLM Pro Finance Suite offers larger models available through commercial licensing.

As of November 21, 2025, the Open ASR Leaderboard remains a pivotal tool for evaluating automatic speech recognition (ASR) models, comparing over 60 models from 18 organizations across 11 datasets. It highlights the growing complexity and specialization in ASR, with new multilingual and long-form transcription tracks added recently. While models combining Conformer encoders with large language model decoders show the best accuracy for English transcription, they are slower, leading to a speed-accuracy tradeoff where simpler CTC and TDT decoders offer faster throughput at the cost of slightly higher error rates. Multilingual models, like OpenAI's Whisper Large v3, provide strong baselines but often sacrifice single-language performance, underscoring the tradeoff between specialization and generalization. In long-form audio tasks, closed-source systems currently outperform open ones, though there is significant potential for innovation within the open-source community. The leaderboard fosters transparent model comparisons and encourages contributions to multilingual ASR, reflecting its role as a community-driven benchmark and a reference point for other language-specific leaderboards.

RapidFire AI, now integrated with Hugging Face's TRL, offers a significant enhancement in fine-tuning and post-training large language models by enabling rapid comparison of multiple configurations without substantial code changes or increased GPU requirements. This tool allows users to concurrently launch multiple configurations on a single GPU and compare them in near real-time, thanks to an innovative adaptive, chunk-based scheduling and execution scheme. The integration can deliver 16-24 times higher experimentation throughput than traditional sequential methods, facilitating faster achievement of optimized evaluation metrics. Additionally, RapidFire AI provides live three-way communication between the user's IDE, a metrics dashboard, and a multi-GPU execution backend, with features like interactive control operations allowing real-time adjustments. The system's design focuses on maximizing GPU utilization and reducing time and resource wastage, with benchmarks showing significant speedups in training times. It offers a user-friendly interface with an MLflow-based dashboard and supports further integrations with other popular dashboards, enhancing the efficiency and effectiveness of machine learning workflows.

NeuTTS-air is a 0.5 billion parameter text-to-speech (TTS) model designed to generate realistic and emotional speech as well as clone voices, although it initially operates slowly on GPUs using transformers. To enhance its performance, Yatharth Sharma optimized the model to generate audio much faster by employing the LMdeploy library, which offers advantages such as simpler installation and low latency compared to alternatives like vllm and sglang. By utilizing advanced techniques such as prefix caching and int8 cache within LMdeploy, Sharma improved batching speed and reduced VRAM usage, despite some minor quality loss. Additionally, the model's codec, neucodec, was replaced with the faster neucodec-distill, which employs more efficient encoders, leading to significant improvements in audio generation speed. Further optimizations involved splitting generated tokens into smaller groups for batch decoding, achieving a remarkable increase in end-to-end processing speed. Future enhancements are planned, including multilingual and multispeaker models, as well as online streaming capabilities, to broaden the model's applications.

AnyLanguageModel, a new Swift package, aims to simplify the integration of local and remote language models for developers on Apple platforms by providing a unified API. It addresses the challenges faced by developers using Apple's Foundation Models, Core ML, or third-party cloud models like OpenAI and Anthropic, which often involve different APIs and integration patterns. By allowing developers to swap their import statements to use AnyLanguageModel without significant code changes, the package reduces the friction in adopting open-source models that run locally. It leverages Swift 6.1 package traits to minimize dependency bloat, enabling developers to include only the necessary backends. Although it extends the capabilities of Apple's Foundation Models, such as adding image support, it risks potential conflicts with future Apple updates. AnyLanguageModel is currently in its pre-1.0 stage, with plans to enhance its features, including tool calling and guided generation, to support complex agentic workflows on Apple platforms.

DeLERP, or Decomposed Linear Interpolation, offers a novel approach to model merging by independently handling the direction and magnitude of neural network weights, addressing limitations of traditional linear interpolation (LERP) and spherical linear interpolation (SLERP). While LERP can weaken a model's representational capacity due to a "norm dip," SLERP, although correcting this, is computationally expensive and privileges the zero vector unnecessarily. DeLERP, inspired by the work of Zheng et al., uses Normalized Linear Interpolation (NLERP) for direction and a max norm strategy for magnitude, ensuring smooth directional transitions without arbitrary geometric constraints. This method preserves the stronger importance signal from either model, maintaining representational capacity with minimal computational overhead. Tested with mergekit, DeLERP demonstrated improvements in capability metrics and alignment when used to merge models, suggesting that maintaining representational capacity can enhance cognitive performance and safety features. While it doesn't directly solve statistical issues like variance collapse, DeLERP's method of preserving magnitude and using geometric direction interpolation may help mitigate such challenges.

In a bid to enhance the efficiency of their 15B reasoning model without compromising its quality, the team at ServiceNow-AI developed a hybrid model named Apriel-H1 by integrating Mamba layers. This process involved a novel insight: distilling the model using high-quality, task-specific data that preserves the reasoning capabilities, rather than relying on pretraining data. By implementing a staged distillation approach, they progressively replaced attention layers with Mamba layers, achieving up to 2.1x throughput with minimal quality loss. The effort culminated in the Apriel-H1-15b-Thinker-SFT model, which maintained reasoning quality across benchmarks. The Fast-LLM framework facilitated this development, offering modularity that allows easy swapping of attention and Mamba layers. While the hybrid model presents significant efficiency gains, deploying it in production requires careful handling due to the current maturity of the tooling, and the team underscores the importance of matching distillation data to the specific capability being preserved.

SQaLe is an extensive text-to-SQL dataset designed to overcome the limitations of existing resources by providing a large, diverse, and realistic foundation for training and evaluating models that convert natural language into SQL queries. Built from over 139,000 database schemas and more than 500,000 validated triples of schema, question, and query, SQaLe reflects real-world schema complexity and is accessible via the Hugging Face Hub for research and model fine-tuning. The dataset addresses the gap in current benchmarks by offering a scale that supports large language models (LLMs) and a realism that mirrors production database environments, with validated SQL queries ensuring consistency with corresponding natural-language questions. SQaLe's creation involved extending schemas sourced from SchemaPile and generating diverse natural-language questions and SQL queries, culminating in a resource that supports the training and evaluation of text-to-SQL models, schema understanding, and benchmark testing in realistic database contexts.

Cogito v2.1, released by the Deep Cogito Team, is a 671 billion parameter Mixture of Experts model that claims to be the most competitive open-weight large language model (LLM) by a US company, excelling in industry benchmarks and internal evaluations. It is designed for enhanced reasoning capabilities, improved instruction following, coding, handling longer queries, multi-turn interactions, and creative tasks, while using fewer tokens compared to similar models. Users can access Cogito v2.1 via a free chat interface at chat.deepcogito.com, where no data is stored, or through various platforms like Huggingface, OpenRouter, Fireworks AI, Together AI, Ollama’s cloud, Baseten, and RunPod. The model is trained with process supervision to strengthen its reasoning process and is available for local deployment with specific hardware requirements, including a quantized version for more efficient serving. The release emphasizes real-world performance and encourages user testing to evaluate its capabilities in practice.

The Pharmome Map is a comprehensive public dataset created by EvE Bio, a non-profit organization under Convergent Research, that aims to transform drug-target interaction modeling by providing detailed measurements of drug activities across a wide range of potential protein targets. This mapping effort seeks to fill the gaps in understanding the effects of drugs on the human body, as pharmaceutical companies traditionally focus on single-target drug development without fully exploring off-target interactions. The dataset, the largest of its kind, is designed to support machine learning applications and includes high-throughput screening data for a library of 1,397 FDA-approved small molecule drugs against key drug target classes such as nuclear receptors, 7-transmembrane receptors (GPCRs), and protein kinases. These data are crucial for advancing fields like AE modeling, polypharmacology, and drug repurposing, offering insights into drug mechanisms, adverse events, and new therapeutic indications. The dataset is structured to include measures of compound activity and potency, and EvE Bio's ongoing work ensures continuous expansion, with new data added regularly, enhancing its utility for pharmacological research and drug development.

Verbatim RAG offers a novel approach to Retrieval-Augmented Generation (RAG) systems by focusing on text extraction rather than generation to eliminate hallucinations commonly introduced by Language Learning Models (LLMs). By constraining the model to extract exact text spans from source documents rather than generating new tokens, Verbatim RAG ensures that every part of the response is directly traceable to the original content, addressing the root cause of factual drift in traditional RAG systems. This method is easily integrated with existing systems like LangChain or LlamaIndex in a few lines of code and operates efficiently on a CPU-only pipeline, avoiding the costs associated with GPU or LLM API calls. Verbatim RAG supports various template management modes for response formatting and provides tools for metadata filtering, index inspection, and debugging. It is particularly suited for applications where precision is crucial, such as in medical, legal, or financial domains, while traditional RAG systems may still be preferable for tasks requiring synthesis across sources or natural language fluency.

Custom kernels are essential for high-performance deep learning, allowing GPU operations tailored to specific workloads, such as image processing or tensor transformations. The process of compiling these kernels for different architectures and integrating them into PyTorch extensions can be challenging, but Hugging Face’s kernel-builder and kernels libraries simplify this by providing support for multiple GPU backends, including ROCm for AMD GPUs. This guide focuses on creating, testing, and sharing ROCm-compatible kernels, using the RadeonFlow GEMM kernel as an example. This kernel is optimized for the AMD Instinct MI300X GPU, using a low-precision FP8 format to enhance throughput and reduce memory bandwidth while maintaining accuracy through per-block scaling. The guide explains how to structure projects, configure build files, and integrate custom kernels as native PyTorch operators, leveraging tools like Nix for reproducibility. Once built, these kernels can be shared on the Hugging Face Hub, making them readily accessible for community use.

Pruna 0.3.0 introduces a significant update to its internal framework, enhancing the flexibility and scalability of algorithm management. This refactor addresses previous limitations where algorithm groups, like cachers or quantizers, were rigidly structured, hindering the integration of new algorithms and their flexible application. Now, classifications act as supplementary metadata, allowing for a modular and composable design that supports new optimization techniques and custom pipelines without structural constraints. The update includes a more streamlined configuration interface, enabling users to define algorithm and hyperparameter settings efficiently using list and dictionary-style configurations. The algorithm execution order is now determined by compatibility rules and constraints, making the system more adaptable and capable of dynamically resolving valid combinations. Users can upgrade to this version without altering existing interfaces, and updated documentation provides guides and tutorials to facilitate the transition.

Amir Mohseni's project explores the development of a router for hybrid Large Language Models (LLMs) that automatically determines whether a task requires reasoning or not, optimizing token usage during computation. The initiative was inspired by OpenAI's advancements in LLMs, particularly the introduction of test-time compute, allowing models to allocate more tokens for complex queries. Mohseni's router, which was built using synthetic data and tested on models like Qwen3-8B, aims to streamline the decision-making process in hybrid models like Qwen3 by automatically choosing between "think" and "no-think" modes based on the task's complexity. This approach notably improves performance over non-thinking baselines while using fewer tokens than full reasoning modes. Despite its promising results, the project has limitations, such as a lack of multilingual and multimodal data and evaluations largely constrained to specific model architectures. The research underscores the potential of automated reasoning mode selection, coinciding with OpenAI's release of GPT-5, which features a built-in router for similar purposes.

Xiaoran Liu's presentation explores the heterogeneous features of attention in long-context large language models (LLMs), focusing on how different attention components across query-key (qk) dimensions play distinct roles. The study leverages the Rotary Position Embedding (RoPE) perspective to explain this heterogeneity, which stems from the use of sinusoidal functions with varying frequencies that define periodicity and monotonicity across dimensions. This leads to innovative applications in length extrapolation, cache optimization, and long-video modeling. The research shows that lower dimensions are stable and handle short-period functions, while upper dimensions manage long-range dependencies, and proposes methods such as FourierAttention for efficient cache compression. The findings contribute to enhancing LLM capabilities, including multi-modality embedding and diffusion-based language models, and emphasize the broader implications for architecture, training, and evaluation in long-context processing.

In this article, the authors discuss their ongoing project to develop a text-to-image foundation model from scratch, focusing on the architectural choices that underpin the model's design. They explore various transformer-based architectures, including DiT, MMDiT, DiT-Air, UViT, and their own custom design, PRX, to evaluate performance in terms of efficiency, scalability, and alignment with text prompts. The PRX architecture emerges as a promising option, balancing speed, memory efficiency, and generative quality, and is introduced alongside a modern text encoder, T5Gemma, which enhances multilingual capabilities and reduces computational demands. The authors also delve into the use of latent space representations and autoencoders like FluxVAE and Deep-Compression Autoencoders to further optimize the training process. The project is open-source, inviting community engagement through platforms like Hugging Face and Discord, as the authors continue to refine their models and prepare for larger-scale training iterations.

Hugging Face and Google Cloud have announced a strategic partnership to enhance the ability of companies to build and customize their own AI using open models. This collaboration aims to leverage Google Cloud's advanced infrastructure and Hugging Face's extensive library of over two million open models to provide seamless AI deployment experiences. The partnership will introduce a CDN Gateway to improve model download speeds and supply chain robustness, and will integrate Google Cloud's unique capabilities into Hugging Face's Inference Endpoints, offering faster deployment times and enhanced security through Google’s Threat Intelligence technologies. This alliance seeks to democratize AI development, making it easier for companies to deploy open models securely within their infrastructure using Google Cloud's leading services like Vertex AI, Google Kubernetes Engine, and Cloud Run, while also ensuring that Hugging Face users benefit from the latest TPU advancements.

The AMD Open Robotics Hackathon, organized by AMD in collaboration with Hugging Face and Data Monsters, is an engaging event designed for students, hobbyists, startup founders, and seasoned engineers to demonstrate their robotics skills through a fast-paced, hands-on competition. Scheduled to take place in Tokyo from December 5-7, 2025, and in Paris from December 12-14, 2025, participants form teams of up to four to tackle two missions: an instructor-led session on setting up the LeRobot development environment using AMD AI solutions, and a two-day project to create an innovative solution to a real-world problem. Participants are encouraged to have strong Linux and Python skills, as well as machine learning proficiency, with hardware provided, including SO-101 robotics kits and AMD Ryzen AI processor-equipped laptops. Prizes will be awarded to the top seven teams in each city, with the first-place team receiving $10,000, based on the creativity, difficulty, ease-of-use, and practicality of their projects. Registration is open to individuals 18 years and older, with specific terms and conditions detailed on the hackathon website.

The article provides an updated overview of the top open-source large language models (LLMs) for 2025, emphasizing the importance of choosing the right model for AI applications in terms of performance, cost, and scalability. It highlights the distinction between "open source" and "open weights," which affects commercial usage due to varying license restrictions. The guide evaluates models based on factors such as task versatility, license permissiveness, local deployment feasibility, context window size, and community adoption. It features models like Llama 4, Qwen 3, and DeepSeek R1, each suited for different tasks such as multilingual understanding, coding, and reasoning, while also providing guidance on deployment methods and license compliance. The article underscores the significance of real-world testing and community support alongside benchmark scores to ensure optimal model selection and deployment.

Photoroom has open-sourced its text-to-image model, PRX, making it available under the Apache 2.0 license through 🤗 Diffusers, with the aim of providing both a robust model and a detailed resource on the training process. The model, which includes a 1.3 billion-parameter version trained on 32 H200 GPUs, is designed to produce high-quality images at resolutions up to 1024 pixels. The release is accompanied by a blog series detailing the training pipeline, including architecture choices, training techniques, and post-training methods, with more updates planned to cover further experiments and refinements. Photoroom encourages community involvement through their Discord server and is actively seeking contributions and feedback. The project showcases extensive experimentation with various architectures, VAEs, and training optimizations, and includes contributions from a diverse team of researchers and engineers.

Optimizers play a crucial role in training large language models like GPT by managing the complex loss landscapes that such models encounter. Stochastic Gradient Descent (SGD), a basic optimization technique, often struggles with issues such as getting stuck in shallow valleys, thrashing in narrow ravines, and making slow progress on plateaus. To address these challenges, advanced optimizers like Momentum and RMSProp were developed, introducing concepts such as accumulated gradients and adaptive learning rates to improve training efficiency. The Adam optimizer combines these ideas, using both momentum and adaptive learning rates to navigate varying terrains effectively, making it the default choice for neural network training despite its high memory cost. Nonetheless, the search for more efficient optimizers continues, with alternatives like the Muon optimizer being explored to reduce memory demands while retaining performance.

RouterArena is an open platform designed to evaluate the performance and efficiency of model routers, which are becoming crucial as the landscape of large language models (LLMs) diversifies from general-purpose giants to more specialized and efficient models. As LLMs expand in variety, selecting the most suitable model for a given task has become a complex challenge, akin to how Google Search routes queries to relevant sources. RouterArena addresses this by providing a comprehensive dataset and a framework for assessing routers across five key metrics: query-answer accuracy, cost efficiency, routing optimality, robustness, and latency. The platform hosts a public leaderboard that allows the community to compare and improve routers, fostering a shift from the "one-model-for-all" approach to a more nuanced ecosystem where intelligent model routing is essential. As the demand for more personalized and efficient AI systems grows, RouterArena aims to standardize and simplify the evaluation process, encouraging innovation in router design and functionality.

SYNTH represents a significant shift in AI training by utilizing a fully generalist synthetic dataset focused on reasoning and skill assimilation rather than traditional large web archives. Developed by Frontier AI labs, SYNTH is built on a foundation of 50,000 vital Wikipedia articles expanded into diverse problem-solving paths, aiming to enhance data efficiency and reasoning capabilities. This synthetic data enables smaller models like Baguettotron and Monad to achieve state-of-the-art results on industry benchmarks with significantly fewer resources. SYNTH employs intricate synthetic pipelines that integrate fine-tuned models and random constraints, fostering a more robust model that can handle various tasks from arithmetic to creative writing across multiple languages. This approach not only improves data efficiency but also emphasizes the importance of context preparation in AI deployment, suggesting that engineering data to understand and enrich domain ontology can significantly enhance the performance of generative models.

Effective prompting for generative vision models is crucial for achieving desired results when using these tools to generate images or videos. The article emphasizes that the quality of the output depends significantly on how well the user crafts prompts, which should guide the model with clear and precise instructions. Key principles include giving direction, using unambiguous language, breaking complex tasks into steps, providing examples, and refining prompts based on output. Different structures are recommended for image generation, video creation, and image editing, each with specific focus elements like subject, action, style, and context. The article also highlights the importance of understanding the model's capabilities and considering ethical aspects such as bias, consent, and data protection. It encourages users to explore and refine their use of vision models while participating in community discussions and staying informed about advancements in AI efficiency.

Kimi K2 Thinking introduces a groundbreaking approach to neural network models by implementing Quantization-Aware Training (QAT) to effectively run a 1 trillion parameter reasoning model at INT4 precision, achieving state-of-the-art performance with doubled generation speed and "lossless" accuracy. Unlike Post-Training Quantization (PTQ), which often results in significant accuracy loss due to its naive approach of compressing a pre-trained model, QAT integrates quantization into the training process, allowing the model to learn robustness to quantization from the outset. This process involves strategic placement of 16 quantization levels, focusing precision where it is most needed, which is particularly beneficial for the Mixture of Experts (MoE) components of the model, where the majority of parameters reside. By training the model to adapt to INT4 precision naturally, QAT prevents the accumulation of errors that typically occur in long reasoning sequences, demonstrating that neural networks can maintain high performance even with highly reduced precision levels.

The article explores an innovative approach to enhancing language models by directly manipulating neural network weights at the binary level, bypassing traditional gradient-based methods. This novel method, encapsulated in the "Tensor Slayer" framework, employs a larger AI system to analyze a model's architecture and weight distributions, generating targeted modification recommendations. The framework enhances the Qwen-0.6B model by strategically modifying 44 tensors, resulting in a 5x improvement in code generation capabilities without additional training or computational resources. The AI-guided approach provides precise, reversible modifications with full transparency, suggesting a potential shift in model optimization towards more accessible, efficient, and transparent methods.

Abliteration is a technique used to modify language models by removing refusal behaviors, and recent advancements have improved its efficacy while maintaining model capabilities. The new method, called norm-preserving biprojected abliteration, focuses on removing only the directional component of refusal behaviors while preserving the weight norms of the model's layers. This approach has shown improved reasoning performance and effective refusal removal in benchmarks when applied to the Gemma3 12B Instruct model. By maintaining the magnitude of weights, the method respects the model's learned importance structure and minimizes unintended consequences. The technique employs a heuristic for layer selection based on signal-to-noise ratio and cosine dissimilarity, ensuring efficient multi-layer interventions that prevent self-repair mechanisms from reintroducing refusal behaviors. This refined approach highlights the potential for removing directionally-encoded safety constraints to unlock latent reasoning capabilities, although it underscores the need for careful consideration of safety implications.

The article details the journey of Jimmy, a nuclear engineering company in France, in developing a Retrieval-Augmented Generation (RAG) system to efficiently search through vast amounts of complex technical documentation. Initially adhering to conventional best practices such as context-aware chunking and hybrid search, the team found these approaches underperformed in their specific context. Instead, simpler methods like naive chunking and dense-only search yielded better results, with AWS Titan V2 embeddings outperforming those from the MTEB leaderboard. The team emphasizes the importance of benchmarking under diverse conditions rather than relying on traditional benchmarks, highlighting that chunk size was not critical for document-level retrieval. They also discuss the choice of Qdrant for vector database storage and the use of Mistral OCR for PDF conversion, cautioning against AWS OpenSearch due to high costs. Overall, the article underscores that best practices in RAG systems should be tested and tailored to specific use cases rather than followed blindly.

ILLUIN Technology, collaborating with NVIDIA, has introduced ViDoRe V3, a comprehensive benchmark for evaluating enterprise document retrieval systems. This benchmark is designed to address challenges in retrieving accurate information from complex, visually-rich documents in real-world scenarios, focusing on multi-modal, enterprise relevance and high data quality. Unlike previous benchmarks, ViDoRe V3 includes human-created and verified annotations and draws from 10 diverse datasets across different industrial domains, with 8 publicly available and 2 kept private. It features 26,000 pages and 3,099 queries translated into six languages, each linked to human-verified retrieval ground truth data. ViDoRe V3 seeks to improve on synthetic data reliance by utilizing a combination of advanced visual language models and human expertise, offering a robust assessment of current retrieval models, which still struggle with multilingual and technical documents, particularly in domains like industrial and energy-related content. The benchmark aims to provide a more realistic and challenging evaluation framework, pushing the boundaries of visual retrieval systems and emphasizing the need for models to synthesize information from multiple pages to address complex queries effectively.

LLaSA has become a prominent framework for LLM-based speech synthesis, and recent efforts have focused on enhancing its prosody and expressiveness through Reinforcement Learning, specifically using Generative Reward Policy Optimization (GRPO). This approach shifts away from traditional maximum likelihood estimation, which often results in flat prosody, by training the model to prioritize qualities such as clarity, expressiveness, and rhythm. The GRPO training pipeline involves generating candidate outputs, scoring them using a reward model that combines word error rate and negative log-likelihood, and adjusting model parameters to favor high-reward sequences. Initial results indicate that GRPO significantly improves semantic consistency and the naturalness of synthesized speech, although speaker similarity gains are inconsistent, and some perceptual aspects of speech remain challenging to capture. Future work aims to develop a learned prosody reward model and incorporate human feedback to further enhance emotional quality, with the ultimate goal of achieving controllable, emotionally expressive multilingual speech.

AI data centers play a crucial role in modern technology by providing the necessary infrastructure for compute, storage, and networking, but they also pose significant environmental challenges due to their high energy and water consumption. These centers have evolved to accommodate AI workloads, shifting from CPU-centric designs to those that leverage GPUs and other AI accelerators, which are more energy-intensive. The global proliferation of data centers, particularly in the United States, has led to increased energy demands, often sourced from fossil fuels, resulting in significant carbon emissions. Additionally, water usage is a concern, as data centers require large amounts of water for cooling and chip manufacturing, putting pressure on local water resources. Efforts to mitigate these impacts include exploring renewable energy sources, nuclear power options, and innovative cooling technologies, but these solutions are not yet widely implemented. As AI continues to grow, there is a pressing need for greater transparency and proactive environmental strategies in the data center industry.

Large language models (LLMs) often default to English for reasoning, even when responding in other languages, which can diminish their effectiveness and consistency in multilingual contexts. Researchers Shan Chen, Jirui Qi, and colleagues explore methods to encourage LLMs to maintain reasoning in the language of the user's query, revealing that small-scale supervised fine-tuning (SFT) can promote language consistency but sometimes at the expense of accuracy. They found that combining SFT with math-focused reinforcement learning (GRPO) can enhance accuracy on complex tasks without reverting to English reasoning, although challenges remain in low-resource languages like Japanese. The study suggests that model merging and targeted fine-tuning can help balance accuracy with language consistency, offering practical strategies for improving multilingual reasoning in AI models.

Since October 2024, compar:IA has provided a platform for users to anonymously compare and vote on the responses of different AI models, contributing to a public dataset for a participatory ranking system. This system, developed in collaboration with the digital regulation expertise center PEReN, aims to enhance transparency and understanding of the generative AI ecosystem based on user preferences rather than technical performance. The ranking, updated weekly and available on platforms like Hugging Face, does not claim to identify the best model but instead reflects collective user preferences, highlighting the ecosystem's dynamics and encouraging model diversity, including open-source options. The ranking's methodology, based on the Bradley-Terry model, emphasizes transparency and reproducibility, with all data and calculations publicly accessible. Observations from the ranking reveal increased competition between proprietary and open-source models and a growing interest in energy-efficient models. The ranking also suggests that perceived performance is not necessarily linked to model size, as user preferences may be influenced by response style rather than factual accuracy. While the ranking offers insights into user preferences, it is intended to complement other forms of evaluation, such as factual, technical, and thematic assessments, to provide a more comprehensive view of AI model performance. Future enhancements may include thematic sub-rankings, analysis by question complexity, and expanded European language support.

Running large Transformer models on mobile and edge devices offers significant advantages in terms of privacy, low latency, and offline usage by processing data directly on the device rather than in the cloud. However, the computational demands of these models present challenges under the constraints of mobile hardware. Techniques such as quantization, knowledge distillation, and model pruning can optimize these models for mobile use. Quantization reduces model size and memory usage by lowering the numerical precision of model weights and activations, while distillation transfers knowledge from a large model to a smaller one, maintaining performance with fewer parameters. Model pruning involves removing unnecessary model parts, which can lead to efficiency gains. The Hugging Face ecosystem, particularly through tools like ONNX, Core ML, and Hugging Face Optimum, facilitates the conversion and optimization of models for deployment on mobile platforms. These methods allow developers to leverage specialized hardware, such as Apple's Neural Engine and Android's NNAPI, for efficient on-device AI processing, making advanced machine learning accessible and practical on mobile devices.

The research explores the potential to train language models with significantly less data while maintaining high performance, specifically focusing on a GPT-2-sized model using only 1 billion tokens compared to the usual 10 billion. Through over 50 experiments, the team identified an optimal pre-training dataset mix of 50% finePDFs, 30% DCLM-baseline, and 20% FineWeb-Edu, which outperformed complex curriculum learning strategies in terms of validation and generalization performance, while also being more efficient. The results demonstrated that thoughtful dataset curation and static mixing strategies can achieve over 90% of the performance of models trained on much larger datasets, effectively challenging the assumption that more data results in better models. This approach not only reduces the computational costs and time but also highlights the importance of dataset quality and diversity in training robust models. The study also provided insights into the pitfalls of curriculum learning, such as catastrophic forgetting and overfitting, reinforcing the benefits of a consistent data distribution throughout the training process.

OpenAI's release of the gpt-oss-safeguard, an open-weight safety reasoning model, marks a significant step towards democratizing AI safety by allowing developers to implement their own safety policies. This initiative signifies a shift from proprietary safety tools locked within major labs to a community-driven approach, emphasizing transparency and adaptability. While the model's technical infrastructure is open, the policies guiding OpenAI's safety systems remain undisclosed, highlighting a gap between technical transparency and normative openness. The release aligns with open innovation principles and recognizes that safety is context-dependent and requires collaboration with diverse stakeholders. It emphasizes the need for open safety benchmarks, community-developed safeguards, and participatory testing frameworks. By fostering partnerships and community involvement, such as through ROOST and various hackathons, the initiative aims to build a resilient and democratic AI ecosystem that aligns with societal values and expectations.