June 2026 Summaries

Dynamic subagents in Deep Agents have been introduced to allow agents to write scripts that orchestrate tool calls, operating within a code interpreter environment. This powerful approach requires secure execution of untrusted code, which is achieved through three design requirements: execution isolation, capability isolation, and durable pauses. Execution isolation ensures that agent-written code cannot compromise the host system, utilizing WebAssembly (WASM) to create a sandboxed environment with a hard boundary. Capability isolation limits what the agent can access, adhering to Meta's rule of two, which restricts agents to no more than two capabilities among accessing sensitive data, exposure to untrusted content, or external communication. Durable pauses allow the program to pause and resume while maintaining its state, using QuickJS within WASM to serialize and restore memory. Two experimental packages, quickjs-rs and langchain-quickjs, support this functionality and are being refined through collaboration with partners as they move toward production readiness.

Memory systems for agents are still in their early stages, with the concept of "memory" varying widely across different applications. A common emerging pattern is "wiki memory," where agents convert raw data into a structured and persistent knowledge layer that is more efficient for agent consumption. This approach involves compressing raw data sources like logs, notes, and documents into a denser, agent-readable format, differing from traditional retrieval-augmented generation (RAG) methods that pull raw data at query time. Notable examples of this pattern include DeepWiki and AutoWiki, which create AI-generated documentation to provide a higher-level understanding of codebases. This compressed knowledge base is maintained by agents and is tailored to be persistent, inspectable, and updated over time, resembling a markdown wiki that sits between users and raw sources. Wikis serve as a durable form of domain knowledge, offering a practical solution to capturing and maintaining essential information across various fields by using files for their inspectability and ease of use with agents.

As agent capabilities advance, evaluations have become more complex, necessitating environments like Harbor, a leading eval harness that provides a reproducible and isolated setting for agent testing. Harbor integrates with tools such as Deep Agents, LangSmith Sandboxes, and LangSmith Observability, enabling agents to operate within clean, isolated environments while being evaluated on tasks that include environment setup, instructions, and evaluation scripts. The integration allows for parallel execution of tasks, enhancing efficiency and ensuring each agent trial runs in its own sandboxed environment without shared states, crucial for assessing agent performance in a controlled manner. Harbor's compatibility with LangSmith offers a comprehensive evaluation stack by logging datasets, experiments, and agent traces, which helps users refine their evaluations and improve agent performance. Users can leverage Harbor's features by setting up their environment with LangSmith credentials and using the eval plugin for a detailed assessment of their agents, ensuring the process remains streamlined and effective.

Dynamic subagents, as introduced by Deep Agents, are designed to enhance the scalability and reliability of AI agents tasked with complex, multi-phase orchestration. Unlike traditional subagents, which are invoked sequentially and struggle at scale, dynamic subagents utilize programmatic orchestration by allowing the main agent to write scripts that manage subagent execution through looping, branching, or concurrency. This approach ensures deterministic coverage and reliable orchestration, addressing challenges like conditional branching and multi-agent fan-outs. By integrating a code interpreter, such as QuickJS, agents can execute these scripts, leading to improved performance in tasks like document summarization or security audits. These dynamic workflows facilitate patterns like classify-and-act, fanout-and-synthesize, and adversarial verification, providing versatility in handling diverse tasks with high confidence and efficiency. Ultimately, dynamic subagents empower agents to break down tasks into manageable units, fostering greater autonomy and adaptability in AI operations.

Candidly's innovative approach to conversational AI involves utilizing a turn-level analysis to enhance the effectiveness of their financial planning assistant, Cait. This method departs from traditional post-conversation evaluations by focusing on real-time state inference and controllable response features to predict and influence conversation outcomes. By employing an Input-Output Hidden Markov Model, Candidly identifies four distinct engagement states—Engaged, Detailed, Guided, and Disengaging—and adjusts Cait's responses to maximize conversation resolution and minimize abandonment. These strategies are validated through a robust pipeline involving data collection, model training, and randomized experiments, all recorded in LangSmith for comprehensive monitoring and evaluation. This framework not only improves Cait's ability to assist users in making significant financial decisions but also demonstrates a scalable model for other multi-turn AI systems seeking to optimize user interactions in real-time.

Prompt caching is a cost-effective feature for running AI agents at scale, offering significant reductions in token costs by storing and reusing snapshots of a model's state after processing a prompt. The Deep Agents harness leverages prompt caching across various model providers to minimize API costs by automatically setting explicit cache breakpoints when supported, and opting into implicit caching otherwise, to maximize cache reads. While different providers offer varied levels of support for features like explicit breakpoints, configurable TTLs, and cache prewarming, Deep Agents ensures that users can switch providers without losing cost-saving benefits. Real-world evaluations with models like claude-haiku-4-5, gpt-5.4-mini, and gemini-3.5-flash have shown token cost reductions ranging from 49% to 80%. As the feature landscape evolves, Deep Agents will integrate new capabilities, while tools like LangSmith provide observability into API costs and caching efficiency to further optimize agent performance.

SmithDB's inverted index implementation facilitates rapid full-text search by constructing, compacting, and querying indexes during data ingestion, allowing new data runs to become searchable within seconds. Index construction is integrated with data ingestion, indexing payloads through a JSON tape based on Apache Arrow's arrow-json crate, and using string interning to optimize sorting. The service uses finite state transducers for term layout and implements a multi-tiered storage approach, leveraging local SSDs for immediate visibility and object storage for durability. At query time, predicates are processed through a unified pipeline that distinguishes between indexed and non-indexed segments without altering the SQL interface, effectively balancing between immediate local reads and comprehensive object-storage reads. The system's design ensures efficient query execution by coalescing GET requests and optimizing memory use during index merging, allowing for sub-second query freshness by treating the local storage tier as an integral part of the index rather than a separate entity.

Zack Reneau-Wedeen, Head of Product at Sierra, discusses the company's innovative approach to conversational AI, which extends beyond customer support to encompass the entire customer lifecycle, in a conversation with Harrison Chase. He emphasizes the importance of using multiple AI models in parallel to leverage their strengths, rather than committing to a single provider, as exemplified by Sierra's use of Claude, Gemini, and GPT-class models for different tasks like reasoning and speech-to-speech. Sierra's outcome-based pricing model aligns incentives by charging based on the value of outcomes rather than usage, advocating for a comprehensive single-agent system per brand to enhance customer experience. Reneau-Wedeen argues that this strategy prevents the fragmentation seen in multi-agent systems, which often result from organizational structures rather than customer needs. The conversation also touches on Sierra's no-code layer, its modular voice architecture, and the lack of a breakout memory company, while highlighting that the solution to AI problems often involves the integration of more AI.

LangSmith has introduced several new features and updates, including an on-call copilot for triage, computer use in Fleet, voice traces, and a new course on LangSmith Deployment. The Fleet On-Call Copilot is a prebuilt agent template designed to help manage alerts and updates, while the computer use feature allows agents to work on an isolated virtual computer. The new interface for voice traces facilitates faster audio debugging with an inline player, and experiment status tracking offers live progress updates. The platform has also integrated with Slack for notifications and introduced open-source tools like RubricMiddleware for self-evaluation by Deep Agents. Additionally, LangSmith has rolled out programmatic subagents to optimize workloads and enhance results. The company is hosting a series of events, workshops, and meetups in various cities, focusing on building and deploying agents with LangChain. Notable success stories include Box's improved agent iteration speed and Harmonic's enhanced customer retention through LangSmith Deployment. Users are encouraged to stay updated through the LangChain blog, changelog, and YouTube channel.

Creating a loop that enables an agent to learn from past actions, known as memory, can significantly enhance user experience by allowing agents to remember and adapt based on previous interactions instead of requiring repeated corrections. The concept involves using a process to identify errors or learning opportunities from past interactions and storing this information in a data structure for future use. Memory is divided into short-term, which is used during current tasks, and long-term, which persists beyond individual interactions and includes facts, preferences, and learned patterns. Implementing this memory loop involves capturing traces of the agent's actions, analyzing them to diagnose issues or patterns, and updating the memory accordingly, using tools like LangSmith Observability for capturing traces, LangSmith Engine for analysis, and LangSmith Context Hub for storing updates. By systematically capturing traces, analyzing them for improvement signals, and updating the memory, agents can benefit from improved behavior over time, making them more efficient and responsive to user needs.

Fleet offers a dual-agent system designed to optimize task management by distinguishing between ad hoc tasks and recurring work. General Purpose Chat is tailored for temporary, broad tasks that require minimal setup and utilize workspace integrations, making it ideal for handling tasks like catching up on Slack threads or reviewing GitHub PRs. This type of agent is temporary, with context isolated to each thread, and is suitable for one-off or exploratory tasks. In contrast, Specialized Agents are configured for recurring tasks with consistent instructions, tools, and memory, allowing them to handle tasks such as weekly planning updates or inbox management. These agents are equipped with durable instructions, scoped tools, and memory that evolves over time, making them efficient for scheduled or event-driven tasks. Fleet's system enables teams to transition from using General Purpose Chat for initial tasks to employing Specialized Agents for repetitive responsibilities, thereby streamlining workflows and reducing reliance on individual memory or prompt rewriting.

Agents automate work in the real world by executing tasks through a series of structured loops, with each loop enhancing the agent's functionality and reliability. At the core, the agent loop involves a model calling tools repeatedly until a task is complete, enabled by frameworks like LangChain's create_agent. To ensure quality, a verification loop checks the output against a rubric, allowing retries with feedback when needed, a process that can increase latency and costs but is crucial for production scenarios. The event-driven loop integrates agents into broader ecosystems, triggering actions through events such as document updates or scheduled tasks, thus supporting continuous operation. The hill climbing loop analyzes traces from agent runs to refine the configuration of the harness, optimizing performance through adjustments and even supporting RL fine-tuning for open-weight models. Human oversight remains essential, particularly for tasks requiring judgment or involving sensitive actions, with frameworks like LangChain facilitating human input at critical junctures in each loop. This multi-layered looping system, or loopcraft, as it's referred to, underscores the importance of embedding agents within ecosystems that evolve and improve autonomously, offering significant competitive advantages to organizations that implement learning loops effectively.

As agents increasingly generate vast amounts of data, understanding their interactions with users becomes crucial, with "perceived error" serving as a valuable metric to evaluate user-agent exchanges. Perceived error focuses on instances where users believe an error has occurred, regardless of the objective accuracy of the agent's response, and is detectable through user corrections and repeated requests. A collaboration with Fireworks led to the fine-tuning of an open-source Qwen judge model to identify perceived errors across applications effectively. By leveraging data from two internal datasets, chat-langchain and Fleet, the research explored the model's ability to generalize perceived error detection across different domains. Fine-tuning demonstrated that models could achieve or exceed frontier performance while remaining cost-effective, with results showing successful transferability and performance on unseen data. The study underscores the potential for fine-tuned models to offer significant cost savings and high accuracy, emphasizing future research directions to enhance trace understanding and the development of specialized models. The refined perceived error model is set to be tested with selected customers, aiming for a broader rollout later to further refine the ability to interpret complex user-agent interactions.

LangChain faced challenges in managing AI usage costs as AI became more integral across the company, with expensive models and coding agents increasing usage unpredictably. To address this, LangChain developed the LangSmith LLM Gateway, a cost-tracking tool integrated into their existing systems, allowing leadership to monitor and control AI spend in real-time. The Gateway enables setting budget limits for organizations, workspaces, users, and API keys, offering flexibility for projects requiring higher usage. Through central implementation and integration with LangSmith's management systems, the Gateway provides visibility into spending, preventing unexpected bills and allowing for traceability and analysis of AI application costs. Lessons from the internal rollout highlighted the need for dynamic model pricing, support for diverse client routes, and workflows around spending limits, leading to improvements in price accuracy and alert systems. Since implementing the Gateway, LangChain has maintained its LLM costs within budget, providing engineering leaders with the tools to manage usage effectively.

AI agents are most effective when they can autonomously write and execute code, but this capability introduces significant security risks, such as prompt injection attacks, which can compromise data and systems. Sandboxes provide a solution by isolating AI-generated code, limiting its permissions, and creating a controlled environment that mitigates these risks. Essential features of a secure sandbox include an isolated filesystem, limited network access, resource limits, controlled reusability, and kernel-level isolation from the host machine. The "Rule of Two," inspired by Simon Willison's work, advises against running agents fully autonomously if they have access to sensitive data, exposure to untrusted content, and the ability to communicate externally. LangSmith offers a managed sandbox solution within its agent engineering platform, providing kernel-level isolation and secure credential handling, integrated with LangChain, LangGraph, and Deep Agents. While sandboxes do not entirely eliminate risks, they significantly reduce them, allowing teams to manage prompt injection risks more confidently.

Box is an intelligent content management platform employed by over 100,000 enterprises to manage their unstructured data with a focus on security and governance. The Box Agent, a component of Box AI, leverages Deep Agents technology to enhance enterprise content search and analysis capabilities, allowing users to perform tasks ranging from simple document queries to complex multi-document analyses across domains. To address enterprise search challenges, Box implemented a parent/child architecture where the Global Agent dynamically creates child agents to efficiently handle diverse tasks, optimizing performance through middleware that manages citations, caching, and context. This architecture supports flexibility with various LLM providers and accelerates the iteration process, enabling rapid deployment of new functionalities. The design anticipates future expansion, aiming to endow the Box Agent with institutional knowledge akin to a seasoned employee, broadening its analytical capabilities and enhancing interaction with internal and external systems.

Nicholas Larus-Stone, the Head of AI at Benchling, discusses the development and impact of Benchling AI, a chat-based intelligence layer for scientific research, in a conversation with Harrison Chase, Co-Founder & CEO of LangChain. Benchling AI, launched in October 2025, utilizes multiple AI models from different providers to enhance data quality and minimize errors, as diverse models tend to make different mistakes. The platform emphasizes the importance of clean data and structured production trace reviews, using user feedback and weekly evaluations to address issues. Larus-Stone notes the significant impact of agents in accelerating scientific workflows, reducing the number of experiments and time needed to achieve results. Benchling's approach also highlights the parallels between understanding large language models and biological processes, suggesting that AI agents are beginning to reshape scientific research methodologies and could potentially lead to novel discoveries in the future.

LangChain's headless tools address the limitations of server-side agent tools by enabling agents to utilize client-side capabilities, such as geolocation, clipboard access, and local memory, directly from the user's environment, whether it's a browser or a desktop application. This advancement allows agents to perform actions that more accurately reflect real application behavior without the need for unnecessary data transfers to a backend server, enhancing both privacy and functionality. By integrating client-side actions into the agent's reasoning loop, headless tools provide a more seamless and dynamic interaction with applications, allowing agents to function effectively in environments where users actively work. This approach not only improves the agent's ability to manage tasks that occur locally but also changes the privacy landscape by enabling data to remain on the device rather than being centralized on a server.

SmithDB is designed to support full-text search and JSON filtering over agent traces with a median latency of 400 ms, despite dealing with large, deeply nested JSON documents stored in object storage. Unlike traditional search engines like Lucene or Tantivy, SmithDB approaches indexing from first principles due to the unique challenges posed by the large payloads of agent traces, which are mainly composed of extensive inputs and outputs that far exceed traditional metadata columns. This requires a compact, efficient inverted index capable of handling the power-law distribution of term frequencies and supporting multiple query modalities. SmithDB's architecture leverages object storage to keep compute stateless, which allows for scalable operations by merely adding nodes. It employs a specialized storage layout for its inverted index, organized by byte-budgeted row groups, to optimize query performance by minimizing object storage requests and efficiently managing term frequencies. The system is engineered to integrate with Apache DataFusion and Vortex, ensuring that search operations can be pushed down through the same scan pipeline as other queries. The design also accommodates the need for compact storage and efficient query execution by using FST-based term dictionaries and block-bitpacked deltas for postings and positions, ensuring high performance even with large, complex data sets.

LangSmith Sandboxes provide a secure and versatile solution for AI agents to execute code by offering each one its own hardware-virtualized microVM, which combines the instant startup of serverless functions with the statefulness of a full machine. This approach addresses the challenges of running untrusted code, which may come from various sources and poses security risks not effectively mitigated by Docker containers due to shared kernel vulnerabilities. By isolating environments, LangSmith Sandboxes enable agents to install packages, run scripts, and maintain state across sessions without compromising infrastructure security. They also enhance GPU efficiency and offer features like snapshots, forks for parallel processing, and pre-warmed blueprints, making them ideal for complex tasks such as coding assistance, data analysis, and CI/CD workflows. Sandboxes empower agents to not only think but act, transforming them from mere assistants into capable entities that can autonomously adapt and execute tasks, as demonstrated by their integration into monday.com's Sidekick AI assistant for advanced user workflows.

LangGraph is a tool designed to enhance the reliability of production agents by providing a structured framework for error handling and fault tolerance. It models agents as a series of discrete steps or nodes, enabling users to manage and recover from various errors such as network failures or API rate limits without restarting entire processes. LangGraph introduces three key primitives for fault tolerance: RetryPolicy for automatic retries with backoff and jitter, TimeoutPolicy for setting time limits on node attempts, and error_handler for executing specific logic when retries fail. These primitives are seamlessly integrated into the workflow engine, allowing users to define fault tolerance configurations directly alongside business logic. This ensures that complex processes, such as a flight booking sequence, can handle failures gracefully through mechanisms like the SAGA pattern, which enables individual step retries and compensatory actions for failed steps. LangGraph's approach to error management significantly reduces boilerplate code and enhances the robustness of agent operations, making it easier to build resilient systems that can handle real-world challenges.

The current shift in software development emphasizes the importance of model neutrality, drawing parallels to the previous transition from on-premises to cloud computing, where vendor lock-in at the tooling layer became a significant issue. Unlike the gradual change experienced during the cloud era, the rise of model-driven development is happening rapidly, with labs racing to control the orchestration layer of business logic through proprietary frameworks, much like cloud providers did with tools like CloudFormation and ARM templates. This has led to a commoditization of tokens and models, where the differentiation between models is diminishing, and the ability to switch between models without vendor constraints becomes crucial. A neutral harness, akin to Terraform in the cloud era, is advocated as essential for ensuring flexibility and avoiding lock-in, allowing businesses to leverage multiple models based on current capabilities, costs, and requirements. Open-source, multi-model support, and profile-awareness are key attributes of such a neutral harness, ensuring that businesses can adapt quickly to the fast-paced advancements in model technology without being tied to a single provider.

Harmonic's platform, originally designed as a venture capital sourcing tool, has evolved into a versatile resource for investors tracking private market trends, with its AI-powered assistant, Scout, serving as a conversational interface for data queries. Initially built with a rigid architecture, Scout's first version faced challenges in adapting to user demands for personalized and complex queries. This prompted a shift to a more flexible system using Deep Agents and LangSmith, significantly enhancing its capabilities and simplifying its architecture. The new version of Scout functions more like a trusted advisor than a traditional search tool, capable of integrating firm-specific context and providing comprehensive insights, such as investment thesis alignment or meeting preparation. This transformation has led to a fourfold increase in user retention and expanded the platform's market reach beyond investors to include innovation teams, recruiters, and others interested in startup ecosystems. Scout's improved user experience and adaptability have positioned it as an industry-standard tool in venture capital and related fields.

Building effective agents involves customizing them to connect with the right context, data, and environment for specific tasks, where the harness plays a critical role by providing the necessary context to the model. The LangChain framework offers a primitive called create_agent, which facilitates the construction of these harnesses by combining models, tools, and system prompts into a working agent. Unlike pre-assembled harnesses, which come with opinionated middleware stacks, create_agent is minimalistic and allows for extensive customization through middleware, which can be integrated at various stages of the agent loop to manage deterministic logic, tool lifecycle, state tracking, and output transformation. This middleware-based approach enables agents to handle complex tasks, prevent context overflow, manage memory, enforce policies, and control costs, while ensuring that the harness is well-fitted to the task's demands. LangChain provides prebuilt middleware for common patterns, but custom middleware can be developed to meet specific use case requirements, allowing for reusable components across different agents.

RubricMiddleware enhances Deep Agents by enabling them to self-evaluate and iterate tasks based on predefined criteria, ensuring that complex tasks are completed accurately. This system introduces a grader sub-agent that reviews the agent's output against a rubric, providing per-criterion feedback and allowing for iterative improvements until all criteria are met or a set iteration limit is reached. This approach mitigates the issues of ambiguous instructions and non-deterministic errors, enhancing output quality and reducing the need for manual intervention by developers. By shifting the responsibility of identifying and correcting errors from developers to the system, RubricMiddleware ensures more reliable task completion, particularly in scenarios where correctness is critical.

Harvey has released LAB, an open-source benchmark designed to evaluate the performance of agents on complex legal work, emphasizing the challenges in verifying the correctness of legal agents due to the domain's complexity and specificity. The LAB benchmark employs a verification method akin to a human reviewer, assessing tasks against a set of predefined criteria using LLM judges. The study explores two methods to enhance verification efficiency: reducing token usage by running verifiers in batch mode and employing cheaper models. Experiments conducted across various legal practice areas demonstrated that batch verification is significantly more cost-effective, although it tends to have lower match rates than per-criterion verification. Models like DeepSeek offer a cost-efficient alternative to frontier models such as Opus, despite discrepancies in label agreement, with targeted prompt tuning reducing false-pass rates. The study emphasizes the potential for open models to enable more affordable and scalable evaluations, challenging the reliance on frontier closed models and advocating for further research into fine-tuning verifiers for specific domains.

Rippling has developed a sophisticated AI-powered workforce management platform to handle its extensive data model spanning HR, IT, finance, and global operations. The AI layer, built on LangChain Deep Agents and LangSmith, facilitates reasoning across thousands of tables and shared concepts that vary by context, overcoming challenges in disambiguation due to overlapping entity names. The multi-agent system includes specialized Deep Agents for querying structured and unstructured data, executing write operations, and a supervisor agent that coordinates tasks. Critical to its success is context engineering, achieved through dynamic skill injection and sandboxed code execution to maintain reliability and audibility. The team implemented a robust evaluation and observability loop using LangSmith for continuous quality improvement, enabling a scalable AI system that serves millions of users globally. Their approach emphasizes building systems familiar to LLMs and creating effective tools for agentic reasoning while maintaining a self-healing evaluation loop to ensure ongoing system health and performance.