December 2024 Summaries

In 2024, the development and implementation of narrowly scoped, highly controllable AI agents gained momentum, with LangGraph emerging as a leading framework for building these systems. Unlike earlier autonomous agent models, LangGraph offers a low-level, transparent approach that appeals to companies seeking more control over their agents' cognitive architectures. Several organizations successfully leveraged LangGraph for various applications, including Unify in account qualification, Komodo Health in healthcare, and AppFolio in property management. Noteworthy examples include LinkedIn's SQL Bot, which enhances data accessibility, Elastic's AI assistant, and Replit's human-in-the-loop and multi-agent system. As these companies shared their insights and experiences through diverse platforms, they contributed to the broader adoption and evolution of AI agents, demonstrating their potential in diverse domains and hinting at further advancements in 2025.

LangSmith Deployment, previously known as LangGraph Platform, introduced custom authentication and resource-level access control for Python deployments, enhancing integration with existing authentication systems and providing granular access patterns for LangGraph applications. This new feature allows for the integration of custom authentication providers and role-based access control (RBAC), ensuring that only authorized users can access specific resources within applications. The system utilizes authentication handlers to validate credentials and identify users, while authorization handlers manage access control, embedding metadata in resources to enforce ownership and access restrictions. The platform supports fine-grained control over various operations, such as threads and assistants, enabling the implementation of custom logic for different resource types. Currently, this feature is available for Python deployments, with plans to extend support to JavaScript in the near future. Users are encouraged to explore the available resources, including video tutorials, guides, and reference documentation, to effectively implement custom authentication in their deployments.

In 2024, the landscape of Large Language Model (LLM) applications continued to evolve, marked by increased adoption of open-source models and a shift towards AI agent applications with complex, multi-step workflows. OpenAI remained the most popular LLM provider among LangSmith users, while Ollama and Groq gained momentum, highlighting a demand for flexible and customizable AI infrastructure. The use of vector stores like Chroma and FAISS remained prevalent, with new entrants like Milvus and MongoDB making strides. Developers increasingly utilized Python for building LLM apps, though interest in JavaScript grew as web-first applications became more common. The LangGraph framework, introduced in March 2024, facilitated the rise of AI agents by allowing more intricate and agentic interactions, evidenced by a significant increase in tool calls within traces. Developers focused on balancing complexity with performance, achieving more sophisticated workflows while managing LLM call efficiency. Quality assurance was a priority, with organizations leveraging LangSmith’s evaluation tools to test and enhance application reliability through automated testing and human feedback loops. Overall, developers embraced complexity and efficiency, paving the way for smarter, more reliable LLM applications.

AppFolio has launched Realm-X Assistant, an AI-powered tool designed to enhance the efficiency of property managers by providing real-time assistance through a conversational interface. The assistant leverages generative AI and is built using LangChain for interoperability, later transitioning to LangGraph to simplify response aggregation and reduce latency. LangGraph's capability of running parallel code branches allows Realm-X to offer related suggestions with minimal delay. To monitor and improve performance, AppFolio employs LangSmith, which aids in debugging, tracking user interactions, and optimizing prompts for precise outputs. This approach has significantly improved Realm-X's performance, particularly in text-to-data functionality, and has enabled AppFolio to maintain high performance as they expand the features and data models available to users. The company prioritizes user feedback and rigorous testing, employing LLM evaluations to ensure the system's accuracy and reliability. Looking forward, AppFolio aims to further enhance Realm-X's performance and reliability through innovations in state management and self-validation loops, continuing to empower property managers with efficient and actionable insights.

LangGraph is designed to facilitate human-in-the-loop workflows in agent-based systems, emphasizing the importance of human intervention in agent tasks. The framework introduces a new function called "interrupt," which allows for pausing the execution of a task graph, enabling human input and modifications before resuming the process. This persistence feature acts as a collaborative workspace, storing the graph's state and allowing for seamless interruption and resumption even in different environments or after extended periods. The goal is to enhance the agent's functionality with various workflows, such as approval processes, state reviews, and multi-turn conversations, making LangGraph a robust tool for integrating human oversight in automated systems. The framework's updates aim to simplify the development of complex human-in-the-loop interactions and demonstrate its utility in real-world applications.

LangGraph, an agent framework powered by an event-driven system, introduces a new tool called Command to facilitate communication in multi-agent systems. This tool enhances the existing graph-based developer experience by allowing more dynamic, edgeless graph representations, addressing the limitations of traditional node and edge structures. Command enables nodes to specify subsequent nodes directly, which is particularly beneficial in dynamic multi-agent architectures where "handoffs" between agents are crucial. By allowing nodes to dynamically control execution flow, Command improves communication and handoffs in hierarchical agent systems. This development aims to enhance the control developers have over agent communication, aligning with LangGraph's mission to optimize multi-agent interactions.

LangSmith now supports ingesting traces in the OpenTelemetry format, an open standard for distributed tracing and observability, enabling developers to instrument and export telemetry data across various programming languages and tools. This update allows LangSmith's API layer to accept OpenTelemetry traces directly, providing a comprehensive view of application performance with unified LLM monitoring and system telemetry. OpenTelemetry defines semantic conventions for attribute names and data across different use cases, and LangSmith specifically supports conventions for generative AI. The platform now accommodates traces in the OpenLLMetry format, which offers out-of-the-box instrumentation for multiple LLM models and frameworks. Users can configure their OpenTelemetry-compatible SDKs to point to LangSmith's OTEL endpoint to ingest traces, and support for other semantic conventions will be added as they develop. Integration examples are provided for OpenTelemetry Python client, OpenLLMetry SDK from Traceloop, and Vercel AI SDK, each offering a method to track and visualize traces in the LangSmith dashboard.

LangSmith has released version 0.2 of its SDKs, introducing several enhancements to improve the developer experience for application evaluation in both Python and TypeScript. Key updates include simplified usage of the evaluate() and aevaluate() methods, improved performance, and expanded documentation. Developers can now create simpler custom evaluators using dict inputs and outputs, and they can run evaluations locally without uploading results, a feature currently in beta for Python. The update also consolidates evaluation methods, making evaluate() the primary method for application and experimental evaluations. Additionally, the Python SDK now features a 30% performance boost for handling large example evaluations, and the documentation has been extensively revamped. This release includes breaking changes, such as altering default concurrency settings and changing the handling of string data arguments to check for UUIDs. Support for Python 3.8 has been discontinued.

LangGraph has introduced semantic search to its BaseStore, enhancing the capabilities of its memory support by allowing retrieval of information based on meaning rather than exact matches. This feature is now available in both the PostgresStore and InMemoryStore, as well as in the LangGraph studio and all LangGraph Platform deployments. Semantic search addresses user demands for more sophisticated retrieval of unstructured information by enabling agents to recall user preferences, learn from past interactions, and maintain consistent knowledge. The implementation involves configuring the server to embed new items through a store configuration, with options to customize embedding logic and specify fields to index. This update is non-breaking for existing users, and LangGraph has provided updated documentation and templates to assist users in integrating semantic search into their workflows.

Cleric is an AI agent designed to assist engineering teams in diagnosing and resolving production issues by conducting parallel investigations on various systems using existing observability tools. It operates similarly to a human engineer by analyzing database metrics, network traffic, application logs, and system resources, communicating findings and seeking guidance through Slack. By leveraging LangSmith's tracing capabilities, Cleric can compare different investigation strategies, aggregate performance metrics, and learn from user feedback to refine its methods and generalize successful approaches across deployments while maintaining privacy. This approach aims to transition more operational responsibilities to autonomous systems, allowing engineering teams to focus on strategic initiatives and product development. Each resolved incident not only improves Cleric's efficiency but also contributes to developing self-healing infrastructure, aligning with the future of engineering that emphasizes product creation over operation.