February 2026 Summaries

Agent-based software, unlike traditional software, presents unique challenges in production monitoring due to its reliance on natural language inputs and large language models (LLMs) that exhibit non-deterministic behavior and prompt sensitivity. Unlike traditional software with finite input spaces and predictable code paths, agents must handle an infinite variety of user queries and perform complex multi-step reasoning, making traditional observability tools insufficient. Effective monitoring requires capturing complete prompt-response pairs, understanding multi-turn contexts, and analyzing decision-making trajectories. Human judgment is crucial for evaluating natural language interactions, but manual review is resource-intensive, leading to the adoption of structured annotation queues and LLMs as proxies for human judgment. Tools like LangSmith provide specialized capabilities for agent observability, helping teams discover usage patterns, evaluate quality continuously, and integrate monitoring with development workflows. These approaches enable cross-functional teams to address the challenges of agent observability, focusing more on monitoring the inputs and outputs of agents rather than just system metrics.

LangSmith Agent Builder is a no-code platform designed for citizen developers to create specialized agents that automate specific tasks, with a strong emphasis on a memory system to enhance user experience. Unlike general-purpose AI agents, LangSmith's agents are tailored for repetitive tasks where memory continuity is crucial, allowing for more efficient and personalized interactions. The memory system is structured using a virtual filesystem supported by Deep Agents, utilizing files stored in a database for ease of access and management. This setup enables agents to update and refine their functionalities through simple configuration and natural language feedback, without requiring domain-specific language expertise. The system prioritizes procedural and semantic memory, while episodic memory is planned for future implementation. Key learnings in building the memory system include the importance of prompting, validation of file types, and maintaining a human-in-the-loop approach for memory updates to prevent errors and potential security risks. The approach not only simplifies the agent-building process but also ensures portability across various platforms, setting the stage for future enhancements such as background memory processes, semantic search, and multi-level memory structures.

Agent observability and evaluation fundamentally differ from traditional software practices due to the non-deterministic nature of AI agents, which perform complex, open-ended tasks. Traditional software debugging relies on deterministic error logs and code paths, but AI agents require tracing to understand their reasoning processes. This shift places emphasis on evaluating agent behavior through runs, traces, and threads, which capture decision-making over numerous steps and interactions. Evaluation levels vary from single-step decision validation to assessing multi-turn conversation flows, with production serving as a key environment for uncovering unpredictable user interactions. As agent behavior emerges in production, offline tests are necessary but insufficient, highlighting the importance of continuous online evaluation. Effective agent development integrates observability and systematic evaluation from the outset, ensuring reliable and adaptable AI agents, with LangSmith offering tools to support this approach.

Agent Builder is a tool that improves through user interaction by retaining feedback, allowing it to function more effectively as a teammate. Built on LangChain's Deep Agents, it utilizes short-term memory for current tasks and long-term memory for persistent data, which can be stored in Markdown files. Users can enhance its performance by instructing the agent to remember successful approaches, using specialized skills for context-specific tasks, and directly editing the agent's memory files for precise updates. These features help in managing the agent's focus and reducing unnecessary information retention, thus preventing confusion and enhancing task efficiency.

Monday.com has developed an evals-driven development framework for its AI Native Enterprise Service Management platform, designed to automate and resolve inquiries across various service departments. By integrating evaluation as a core component from the outset, the company has significantly accelerated feedback loops, enabling comprehensive testing across numerous examples in minutes rather than hours. The framework employs a dual-layered evaluation approach: offline evaluations act as a safety net, using curated datasets to ensure core logic and specific edge cases are robust, while online evaluations provide continuous quality monitoring in real-time production environments. The evaluations are managed as version-controlled code, using GitOps-style CI/CD deployment, which enhances agent observability and ensures high-quality AI interactions. The platform's architecture allows for parallel and concurrent testing, drastically improving evaluation speed and efficiency. This structured approach to evaluations, with tools like LangSmith and Vitest, reflects a commitment to rigorous testing standards akin to production code management, ensuring the AI service workforce remains reliable and adaptable to various enterprise service management use cases.

LangSmith Agent Builder has been significantly updated to enhance user interaction by incorporating a central "Chat" agent that operates like a teammate, allowing for ad hoc tasks and the creation of specialized agents. The update includes features such as the ability to directly upload files like CSVs, images, and text files into the chat for various tasks, and manage all tools from a single registry with admin controls. Users can now transform any conversational task into a recurring agent with a single click, and the Chat agent can access all connected tools such as Slack, Gmail, and others to perform tasks seamlessly. This allows for complex, multi-step operations without needing separate dedicated agents, and every conversation is saved for future use, enabling users to create agents from past interactions. Additionally, the update simplifies tool management, ensuring that only workspace administrators can add new tools, keeping the system organized and secure.

Harness engineering at LangChain plays a critical role in optimizing coding agents' performance by refining systems around a fixed model, specifically the gpt-5.2-codex, rather than altering the model itself. By focusing on elements like system prompts, tools, and middleware, the team improved their agent's score from 52.8% to 66.5% on the Terminal Bench 2.0 benchmark, which evaluates agentic coding across various tasks. Key strategies include employing automated trace analysis to identify and rectify errors, enhancing agents' self-verification capabilities, and optimizing context delivery. The approach underscores the importance of guiding agents to write testable code and managing computational resources efficiently. This iterative improvement process involves a combination of automated tools and human interventions to ensure agents do not become trapped in unproductive loops and can autonomously adjust their reasoning and problem-solving strategies. The exploration of multi-model systems and memory primitives for continual learning suggests promising future directions for harness engineering, with an emphasis on context engineering, self-verification, and adaptive reasoning as fundamental principles.

As large language models (LLMs) continue to improve, the relevance and evolution of agent frameworks remain a critical topic of discussion, prompting a re-evaluation of their necessity alongside increasingly advanced models. Despite criticisms of their usefulness, especially as the AI landscape rapidly changes, agent frameworks have proven essential in encoding best practices, reducing boilerplate code, and facilitating faster, higher-quality development, especially for teams. The evolution from simple chaining methods to more complex orchestration and autonomous tool-calling-in-a-loop agents illustrates their adaptability to diverse use cases. LangSmith, independent of LangChain's open source, was developed to ensure quality and observability across various frameworks, supporting OpenTelemetry-based tracing for diverse integrations, even for those not using any specific framework. As agent engineering continues to merge building with testing, understanding agent behavior through traces becomes indispensable, underscoring the importance of debugging, testing, and monitoring in the development process.

Interrupt - The Agent Conference by LangChain is a gathering designed for developers, product leaders, researchers, and founders to share insights on building and implementing agents in production. Scheduled for May 13-14 in San Francisco, the event promises keynotes from influential figures like Harrison Chase and Andrew Ng, focusing on future developments in agent technology. Attendees can expect to gain practical lessons from AI teams at companies like Clay and Monday.com, explore new LangChain products, and participate in hands-on workshops to enhance their skills. The conference also provides opportunities for open discussions and networking with sponsors such as Cisco and Oracle, aiming to foster collaboration among participants. Tickets are currently available, with a limited seating capacity encouraging early registration.

As more agents require workspaces to execute code, install packages, and access files, sandboxes provide the necessary isolation to prevent unauthorized access to credentials and networks. Two primary architecture patterns for integrating agents with sandboxes are identified: running the agent within the sandbox, which offers a development-like environment with tight coupling between the agent and its environment but poses security risks and infrastructure challenges; and using the sandbox as a tool, allowing the agent to run locally or on a server while executing code remotely, facilitating easier updates and maintaining API key security while potentially incurring network latency issues. Each pattern offers unique benefits and trade-offs, depending on the specific needs of agent development, such as mirroring local development or prioritizing quick iterations and secure API management. Deepagents, an open-source agent framework, supports both approaches, enabling users to choose the best fit for their requirements.

LangSmith, an agent engineering platform from LangChain, is now available in the Google Cloud Marketplace, allowing Google Cloud customers to seamlessly integrate and manage LangSmith through their existing accounts, benefiting from simplified billing and procurement processes. LangSmith, which is already trusted by prominent engineering teams, enhances the development of AI applications by offering a unified platform for debugging, testing, deploying, and monitoring, with features like observability, evaluation, and deployment capabilities. It integrates with Google Cloud to support multiple models and services, providing enterprise teams with a versatile and scalable solution for building, testing, and deploying AI applications. The platform's availability through Google Cloud Marketplace facilitates consolidated billing, allows purchases to count towards existing Google Cloud commitments, and streamlines procurement processes, supporting various deployment configurations to meet diverse security and compliance needs. LangChain, known for its open-source frameworks like LangGraph and LangChain, collaborates closely with Google, enhancing AI workflows with integrations such as the Gemini model and Agent2Agent Protocol, further extending the capabilities of LangSmith for a broad range of AI applications.