February 2025 Summaries

LangGraph has emerged as a promising framework for building AI agents, favored by companies like Replit, Klarna, LinkedIn, and Uber, due to its low-level approach that eschews hidden prompts and enforced cognitive architectures, making it production-ready and distinct from other frameworks. While acknowledging the benefits of higher-level abstractions for ease of use and experimentation, LangGraph has introduced prebuilt agents to balance accessibility with flexibility. These agents, packaged separately in LangGraph 0.3, include Trustcall for structured extraction, LangGraph Supervisor for multi-agent architecture, LangMem for long-term memory, and LangGraph Swarm for swarm architecture, all available in Python and JavaScript. This initiative aims to stimulate a community-driven collection of prebuilt agents, with guidelines provided for developers to create and register their own packages, drawing inspiration from the successful integration model of LangChain, which boasts over 700 community-maintained integrations.

MUFG Bank, Japan's largest financial institution, has addressed the challenge of data overload in its Global Capital Markets Division by leveraging Generative AI (GenAI) to enhance efficiency in the FX & Derivative Sales team. The AI/ML team at MUFG implemented LangChain to streamline data extraction and automate the creation of client presentations, significantly reducing the time required from several hours to just 3-5 minutes. This system, which utilizes retrieval-augmented generation (RAG) and few-shot prompting, enables sales teams, including less experienced members, to generate comprehensive financial insights and recommendations. As a result, MUFG has achieved a tenfold increase in the number of corporate clients receiving tailored financial advice, leading to faster deal execution. The success of this initiative was facilitated by the LangChain programming library, first using Python for research and development and then transitioning to TypeScript for production, enabling dynamic content filtering and retrieval. MUFG plans to further refine its GenAI applications by incorporating broader data sources and exploring advanced AI architectures to continue enhancing sales intelligence and decision-making.

Evaluations (evals) are essential for deploying reliable LLM-powered applications, providing systematic methods to assess LLM output quality based on specific criteria. The newly introduced packages, openevals and agentevals, offer a set of evaluators and a framework to simplify the process of building evaluations from scratch. Evals involve two components: the data being evaluated and the metrics used for evaluation, both of which significantly impact the reflection of real-world usage. The packages focus on common evaluation types, including LLM-as-a-judge evals for natural language outputs and structured data evaluations for extracting or generating structured content. Additionally, agent evaluations assess the sequence of actions taken by an agent to complete tasks. Openevals and agentevals provide tools to customize evaluations, incorporate human preferences, and ensure consistency, while LangSmith offers capabilities for tracking and sharing evaluation results. Future plans include expanding the libraries with more specific evaluators and encouraging community contributions through GitHub.

Onyx, an AI assistant, enhances productivity by connecting documented knowledge from various sources using large language models (LLMs) to answer complex enterprise-related questions. They have developed an advanced search method called Agent Search, which deconstructs broad questions into narrower sub-questions, composes initial answers, and refines them based on relevant documents and learned facts. The blog outlines their experience in implementing this flow using LangGraph, an open-source agentic framework, which offers features like parallelism and streaming. Onyx selected LangGraph due to its alignment with their logical flow and its potential for future enhancements, despite some concerns about third-party dependencies and state management. Their implementation emphasizes code organization, state management, and the use of subgraphs for parallel processing, with plans to further develop their system for more agentic capabilities in the future.

LangMem SDK is a new library designed to enhance AI agents by incorporating long-term memory capabilities, allowing them to learn and improve over time. The SDK offers tools to extract information from conversations, optimize agent behavior, and maintain memories of behaviors, facts, and events, integrating seamlessly with any storage system and agent framework. A managed service is also available, providing additional long-term memory results for free. LangMem supports different memory types, including semantic memory for facts and knowledge, procedural memory for behavior patterns, and episodic memory for past experiences, each serving distinct functions to build adaptive agents. The SDK emphasizes the importance of mapping agent capabilities to specific memory types and questions like which behaviors should adapt or remain fixed, what knowledge to track, and when memories should be recalled. Privacy and performance considerations are addressed by scoping memory to users or teams, while integration with LangGraph ensures effective short-term memory management. LangMem aims to make AI experiences smarter and more personalized, building on prior work with LangGraph's memory layer, and it invites users to explore custom memory systems and join their team in advancing adaptive AI agents.

Klarna, a leader in fintech, has transformed global commerce through its consumer-centric, AI-powered payment and shopping solutions, boasting over 85 million active users and 2.5 million daily transactions. The company addresses the challenge of scaling customer support and handling escalations by utilizing an AI Assistant powered by LangGraph and LangSmith, which has significantly improved efficiency by performing tasks equivalent to 700 full-time staff. This AI Assistant, enhanced by controllable agent architecture, context-aware intelligence, and test-driven development, has reduced average customer query resolution time by 80% and automated approximately 70% of repetitive support tasks, allowing customer service agents to focus on complex interactions. The advanced prompt optimization and meta-prompting features have further improved the accuracy of root cause identification, leading to a notable reduction in customer escalations and empowering Klarna's teams to deliver superior customer experiences globally.

The study explores the effectiveness of a single ReAct agent architecture in handling tasks across multiple domains, focusing on Calendar Scheduling and Customer Support. It aims to determine how increasing the number of domains affects the agent's performance, specifically when tasked with following instructions and using tools within these domains. The research evaluates several models, including claude-3.5-sonnet, o1, o3-mini, gpt-4o, and llama-3.3-70B, using 30 tasks for each domain, run three times to account for non-deterministic behavior. Results indicate that as more context and tools are introduced, agent performance declines, particularly in tasks that require longer tool-calling trajectories. Models like o1, o3-mini, and claude-3.5-sonnet generally outperformed gpt-4o and llama-3.3-70B, although o3-mini showed a sharp performance drop with increased context. The study suggests that multi-agent architectures may offer improvements over single ReAct agents when managing a large number of domains, and plans to explore this further alongside cross-domain tasks and more complex trajectories.

Vizient is transforming healthcare data access and analysis through its GenAI platform, which enables healthcare providers of varying sizes to unify and query disparate datasets, facilitating better decision-making in supply chain management and clinical outcomes. The platform addresses challenges of data silos and operational inefficiencies, providing instant data-backed insights into questions such as the effectiveness of ambulatory investments or cost-efficiency of care. Vizient employs LangGraph to orchestrate a hierarchical multi-agent system, enhancing reliability and workflow management, while LangSmith offers observability and prompt management capabilities, allowing teams to easily pinpoint and resolve issues and iterate on prompt logic. As Vizient expands its GenAI platform, it remains committed to maintaining trust, security, and innovation, empowering healthcare providers to derive actionable insights even without data expertise.

Infor, a leading enterprise software company, offers cloud-based multi-tenant solutions tailored to various industries and has integrated generative AI into its services to enhance product functionality and customer experience. By transitioning its chat assistant, Coleman DA, to a more flexible, LLM-powered platform, Infor enables complex query handling, dynamic content generation, and seamless integration with machine learning models. Utilizing tools like LangChain, LangGraph, and AWS Bedrock, Infor has developed a scalable AI platform that provides rich industry knowledge and improved customer support. Key components of this platform include GenAI embedded experiences, a GenAI Knowledge Hub, and a GenAI Assistant, which enhance user interactions and streamline processes. The integration of LangSmith ensures strong observability, compliance, and performance tracking across Infor's generative AI initiatives, allowing for efficient troubleshooting and optimization of AI-generated outputs. As Infor continues to innovate, the company is committed to empowering customers by leveraging AI to enhance business operations and manage complex workflows through advanced multi-agent interactions.

In 2024, companies like Uber, LinkedIn, and Replit increasingly adopted LangGraph to develop specialized AI agents tailored to specific business needs, addressing challenges such as reliability, observability, and control in production environments. LangGraph, a customizable and low-level agent framework, offers flexibility and reliability, allowing companies to build scalable systems with human-in-the-loop capabilities and enhanced transparency. This framework helps overcome hurdles like the unpredictability of LLMs, complexity of orchestration, and limitations in observability and debugging, which are crucial for deploying AI agents effectively. As companies continue to leverage LangGraph, it is set to drive the growth of AI agent adoption in 2025 by enabling the creation of reliable, production-ready AI solutions.

Interrupt is LangChain's first conference dedicated to exploring the potential, practical applications, and challenges of AI agents, taking place on May 13-14 at The Midway in San Francisco. The event is designed for software engineers, AI/ML practitioners, data scientists, researchers, and engineering leaders who are involved in building AI agents. With a focus on transforming prototype applications into production-ready large language model (LLM) solutions, the conference will feature hands-on workshops, keynotes, expert talks, and networking opportunities. Attendees will learn best practices, explore innovative use cases, and collaborate with industry pioneers such as Andrew Ng and companies like Cognition Labs and Replit. The event aims to foster knowledge exchange and propel the AI agent ecosystem forward, with early-bird tickets available for both the pre-conference workshops on May 13 and the main conference on May 14.