July 2026 Summaries

Agent and model evaluations in the Agent Platform are now generally available, providing tools to measure and compare the quality of agents and models during development and after launch using consistent metrics. The platform offers over 20 pre-built metrics for various tasks, such as summarization and translation, and allows users to define custom metrics. Experiments can be run client- or server-side, and the system supports issue clustering to interpret evaluation failures efficiently. Online monitors enable continuous evaluation on live production traffic, offering score-over-time charts and drift alerts. The evaluation service includes a flexible UI for defining metrics and reviewing results, with all experiment artifacts stored in Cloud Storage for transparency and compliance. Additionally, the platform supports case generation and simulation to create comprehensive evaluation datasets, while pricing is based on model calls and Cloud Storage use. Integration with existing tools is facilitated through the Agent Platform SDK and agents-cli, allowing seamless evaluation processes for agents built on ADK and other frameworks.

Large language model-based agents are facing challenges in managing prompt context size as their tasks grow more complex, leading to inefficiencies and inaccuracies. To address this, Genkit has introduced Agent Skills for TypeScript, Go, Dart, and Python, which package specialized expertise into discoverable capabilities activated only when necessary. This method, based on progressive disclosure, optimizes token usage by loading only essential skill metadata initially and detailed instructions when required. Skills are organized with a SKILL.md file containing metadata and instructions, and can include additional resources like scripts. The Genkit middleware architecture supports these skills by dynamically loading them based on incoming prompts, enhancing the agent's ability to execute tasks accurately. The application of Agent Skills is demonstrated through a Go-based recipe generation and a multi-modal art restoration application, showcasing their utility in handling non-deterministic tasks while maintaining resource efficiency.

Understanding the performance of Tensor Processing Units (TPUs) requires empirical evaluations beyond theoretical specifications, utilizing a microbenchmark suite to assess real-world performance across various architectural environments and workloads. This suite evaluates TPUs by segmenting performance into key functional areas such as network, compute, high-bandwidth memory, host transfer, and attention mechanisms, providing granular insights into whether the devices meet their theoretical capabilities and identifying architecture-specific bottlenecks. By establishing a "Speed-of-Light" baseline, these microbenchmarks transform performance optimization into an empirical discipline, using the Roofline model to classify bottlenecks as compute-bound, memory-bound, or network-bound, and guiding optimization strategies like kernel selection, sharding, and rematerialization. The insights gained from these benchmarks are crucial for predictive modeling and large-scale deployment optimization, as demonstrated in a case study on Ironwood TPU 7x, where microbenchmark data significantly improved the performance of a Mixture-of-Experts training workload.

The second part of the guide on using Ray with TPUs explores the AI libraries necessary for building effective workloads, focusing on Ray Serve, Ray Data, and Ray Train. It highlights the importance of the topology field in Ray Serve for deploying multi-host models on TPU slices, which prevents deployment issues by ensuring tensor-parallel model workers remain on a shared ICI mesh. Ray Data is discussed in terms of its iter_jax_batches() function, which optimizes data flow into TPUs by delivering JAX-native batches directly, thus avoiding bottlenecks. Meanwhile, Ray Train is simplified by JaxTrainer, which manages distributed training with JAX on TPUs, removing the need for complex configuration and providing features like checkpointing and fault tolerance. The guide also mentions the availability of official TPU Docker images and enhanced Ray Dashboard metrics for monitoring TPU usage, emphasizing the ease of transitioning from GPU to TPU usage with Ray while teasing future enhancements in TPU support.

The rapid evolution of LLM alignment has shifted towards dynamic agentic workflows, where models execute complex multi-step reasoning and interact with intricate environments. This change poses challenges in training reasoning agents, particularly in maintaining efficient hardware utilization and overcoming infrastructure bottlenecks. Google's Tunix library addresses these challenges by introducing asynchronous rollouts and a barrier-free pipelining architecture that maximizes TPU throughput while decoupling rollout and training processes. Tunix offers composable agent and environment abstractions, allowing seamless integration with open-source environments and facilitating easy customization without extensive code modifications. Additionally, it enhances observability with lightweight RL-specific profiling metrics to identify and resolve system bottlenecks. This positions Tunix as a leading framework for agentic RL, offering a high-performance foundation for developing advanced reasoning agents and integrating seamlessly with the JAX/TPU ecosystem.

Ray, a distributed-computing framework, now supports Google Cloud TPUs as a first-class accelerator, enhancing its capability to schedule tasks across clusters using TPUs similarly to GPUs. This integration is facilitated by Google's Kubernetes Engine (GKE), which provisions TPU slices—fixed groups of interconnected TPU chips—and labels them for Ray's use. With this setup, Ray Core can reserve entire TPU slices atomically, ensuring that distributed tasks are executed efficiently without manual placement coding. The public API for TPU support in Ray, though marked as alpha, allows developers to leverage slice placement through a single function call, making it easier to deploy and manage AI workloads. This new capability promises seamless scaling of Python applications on TPUs using familiar Ray AI libraries, and further exploration in Part 2 will demonstrate practical applications like serving large language models and training with JaxTrainer.

Generative AI is advancing to perform complex tasks, necessitating a high level of factual accuracy, and Google Cloud has partnered with Parallel Web Systems to enhance this capability on the Gemini Enterprise Agent Platform. By integrating Parallel Web Systems as a native web grounding provider, developers can now anchor Gemini models in real-time, verifiable web results, ensuring accurate and reliable information with precise citations. This integration, available through the Google Cloud Marketplace, allows enterprises to utilize Parallel’s Search API, which delivers structured, LLM-optimized results, enabling production-grade autonomous agent workflows at scale. With this partnership, enterprises gain expanded licensing and architectural flexibility, facilitating the creation of specialized workflows such as catalog enrichment, autonomous agents, and multi-agent orchestration. By offering a streamlined integration within existing Google Cloud environments, the partnership supports complex architectures and the development of sophisticated AI agents that can seamlessly execute programmatic tasks and enrich internal datasets with real-time web data.

Conductor has evolved from a Gemini CLI extension into a plugin, enhancing its ability to facilitate spec-driven development (SDD) by enabling more fluid and conversational interactions. This transition allows Conductor to integrate seamlessly with various tools like Antigravity CLI and Claude, making it a portable and ecosystem-wide capability. The plugin maintains the benefits of persistent markdown files for project documentation while offering a dynamic, AI-assisted workflow that simplifies task completion and preserves project context. This evolution promises improved success rates in complex tasks and ensures backward compatibility with existing commands and plans. The Conductor Plugin aims to make AI development safer and more architecturally sound by providing a more intuitive and adaptable interface for users.

As AI agents transition from development to production, the complexity of their monolithic system prompts becomes a significant challenge, leading to issues such as obscured blast radius, copy-paste drift, and deferred runtime errors. These challenges can be mitigated by treating prompts as software artifacts, using modular skill files to separate concerns and improve maintainability. This approach allows for deterministic builds, static validation, and CI/CD integration, ensuring that every change can be tested, audited, and diffed before deployment. A templating system can help compose shared instructions and inject environment-specific values, while a production-grade transpiler catches errors before runtime, reinforcing reliability through dependency graphs and drift checking. By adopting progressive disclosure, agents load only the necessary skill modules for specific tasks, enhancing performance and reducing context exhaustion. This modular system enables agents to propose improvements to their own logic, subject to the same validation and review processes as any software change, thereby ensuring the high reliability standards required for AI agents involved in critical workflows.

Deploying and serving the Mixture-of-Experts (MoE) model Qwen3.5-397B on specialized hardware like the Ironwood TPU v7x poses significant engineering challenges due to its massive weight footprint and complex architecture. The model's novel components, such as Gated DeltaNet (GDN) linear attention and Attention Data Parallelism, require a modular, model-agnostic optimization strategy to manage these complexities efficiently. By decomposing the model into independent building blocks and using pre-optimized modules, engineers can achieve significant performance improvements in inference workloads. Between April and June 2026, these optimizations resulted in a 3.1x performance increase for decode-heavy and a 4.7x increase for prefill-heavy workloads. Furthermore, by integrating these optimizations into open-source frameworks like vLLM and SGLang, legacy software barriers are minimized, supporting seamless migration paths for enterprise workloads. Through systematic hardware-aware sharding and custom kernel developments using the JAX/Pallas language, the team achieved high efficiency, extracting up to 82.4% and 79.6% of the TPU’s theoretical limits for compute-bound and memory-bound workloads, respectively. This approach not only optimizes the Qwen 3.5 model but also establishes a reusable software stack, enhancing TPU capabilities for future MoE architectures.

At the Google I/O India event, a joint showcase with the Google Pixel team highlighted the capabilities of Google Tensor’s custom System-on-Chip (SoC) and advanced Tensor Processing Unit (TPU) in driving private on-device AI for the Google Pixel 10 family. The event introduced the Gemma 4 E2B model for TPU, designed for offline privacy and efficient edge AI applications, allowing functionalities such as personal assistance and core phone operations to be performed offline using private voice or text commands. The technology demonstrated rich offline multimodal interactions, enabling tasks such as AI chat, object identification, and audio transcription without internet access, and showcased real-world applications in retail and automotive diagnostics. Developers were encouraged to explore the Tensor SDK and its resources for building secure, edge-based AI on Pixel devices, with acknowledgments given to the collaborative efforts of various teams for making the event possible.

LiteRT.js is a JavaScript binding of LiteRT designed to run AI models directly in web browsers, offering enhanced user privacy, zero server costs, and low latency by performing ML and AI model inference locally. It serves as an evolution from TensorFlow.js, providing smoother deployment for existing .tflite models by leveraging WebAssembly and native hardware acceleration, including XNNPACK for CPU, ML Drift for GPU, and the emerging WebNN for NPUs. The initial release includes a new npm package and demos, showcasing integration capabilities for web developers using JavaScript or TypeScript for tasks like text generation, object detection, and audio processing. LiteRT.js supports PyTorch conversion, tailored quantization, and high-performance inference across CPU, GPU, and NPU backends, delivering significant speed improvements over other web runtimes. The framework's integration with Ultralytics' YOLO models demonstrates its real-world application in real-time object detection and image processing. As LiteRT.js continues to develop, future plans include advancing WebNN integration and enhancing support for on-device generative AI.

On May 23, 2026, Google Developer Experts gathered at Sonoma Raceway to test an AI-powered race coach designed to provide drivers with real-time, actionable advice using Google’s Antigravity and Gemini technologies. This initiative aimed to close the AI trust gap by grounding AI architecture in physics and real-time verification, exemplified by the system identifying a new throttle application zone mid-corner in Turn 2, which improved lap times. The project utilized a sophisticated technology stack, including Google Cloud Platform, Agent Development Kit, and Jetpack Compose, to process and analyze telemetry data, allowing for strategic insights and immediate coaching. The event also highlighted the technical breakthrough of activating the Pixel 10 TPU for real-time reliability and demonstrated the potential of agentic orchestration in mission-critical domains beyond racing. Following the success at Sonoma, the initiative plans to advance its architecture by testing in new environments like Interlagos, Brazil, further bridging the AI Trust Gap.

The text explores the concept of elastic training in the context of distributed machine learning, particularly focusing on the use of the JAX AI stack, MaxText, and Pathways to train large language models on Cloud TPUs. Elastic training is presented as a solution to the fragility of traditional distributed training, where a single machine failure can halt an entire multi-node operation and necessitate a complete restart from the last checkpoint. The article demonstrates how elastic training allows for in-place recovery without restarting the job, by catching failures as exceptions within a single Python process and utilizing components like MaxText, Pathways, and Orbax for seamless recovery. This process involves resuming training from the last viable checkpoint or continuing on surviving slices while replacements come online, thus minimizing downtime and loss of progress. The discussion includes a demonstration of the recovery process, highlighting the efficiency of elastic training in reducing restart costs and maintaining training continuity, and provides insights into scaling the approach, as well as future advancements like snapshot-based elasticity for improved recovery times.

Genkit introduces an open-source framework designed to simplify the development of full-stack, AI-powered applications through its Agents API, accommodating languages such as TypeScript, Go, Dart, and Python. The framework addresses the complexity of building multi-turn conversational features by consolidating essential components like message history, tool loops, and state management into a single interface, allowing developers to focus on the unique aspects of their applications. It offers flexibility in managing conversation continuity either on the server or client-side, providing options for persistence through tools like Firestore or custom implementations. Genkit supports seamless integration with HTTP endpoints and offers a JavaScript client for web-based applications, facilitating a unified chat interface. With capabilities for streaming, human-in-the-loop interactions, and task delegation among specialized agents, Genkit aims to streamline the development of conversational AI applications while offering tools for testing and debugging within its Developer UI. The Agents API is currently in beta and welcomes community feedback to refine its offerings.

Transitioning AI agents from prototype to production in enterprise settings presents challenges such as infinite loops, hallucinations, and failure without clear exceptions. Traditional methods focusing on model functionality, like guardrails and skills, have limitations, necessitating deterministic control over application flow for reliable production. Large language models, though capable, are inefficient for tasks like routing and error handling compared to traditional code. ADK 2.0 addresses these challenges by introducing a structured workflow runtime and task-collaboration model, blending the flexibility of AI with the reliability of deterministic execution. This new version enhances the capabilities of its predecessor by allowing developers to create workflows that separate execution routing from language processing, reducing token consumption and latency. It also provides a dynamic, modular approach to handling complex business logic, ensuring secure execution pathways and structured multi-agent collaboration, ultimately offering a balanced solution for building scalable, trustworthy AI applications.

The launch of the Google Cloud Workbench Notebooks extension for VS Code provides data scientists and developers with an integrated tool that combines the convenience of a local IDE with the robust capabilities of Google Cloud's infrastructure. This extension facilitates a seamless transition from local experimentation to cloud-based computation by enabling users to manage both their code and cloud-based notebooks within a single interface, thus streamlining the machine learning lifecycle. The integration eliminates the need for context switching, allowing for efficient connection and scaling between local and cloud environments. Furthermore, the extension supports open-source innovation, encouraging community contributions and transparency, and aims to simplify the process of leveraging Google Cloud's AI-optimized infrastructure for data science projects.