March 2023 Summaries

LangChain is evolving its framework to accommodate a broader range of retrieval methods beyond the LangChain VectorDB object, aimed at enhancing flexibility and innovation in constructing personalized data-driven chatbots. This adjustment allows for the integration of externally constructed retrievers, promoting experimentation with alternative retrieval methods such as hybrid search, while maintaining backward compatibility with existing chains. The new approach involves replacing VectorDBQA chains with RetrievalQA chains, utilizing a more generic "Retriever" interface, which requires only a get_relevant_documents method, thus simplifying integration with various retrieval mechanisms. LangChain has also incorporated OpenAI’s ChatGPT Retrieval Plugin, exemplifying the new flexibility and encouraging the development of novel retrieval techniques. The initiative seeks to improve the process of question-answering over personalized data and enhance memory capabilities in AI applications by facilitating the retrieval of relevant documents using diverse retrieval strategies.

The integration of Zapier's new NLA API into LangChain allows users to access over 5,000 apps and 20,000 actions available on Zapier's platform through a natural language interface, significantly expanding the capabilities of LangChain agents and chains. This integration simplifies the process by handling API authentication and translating natural language inputs into API calls, offering both API Key and OAuth options for different use cases. For developers, a Toolkit is provided in Python and TypeScript, enabling easy setup by retrieving an NLA API Key, setting an environment variable, and initializing an agent. The ZapierToolkit automatically registers enabled Zapier actions as tools, and users can manually register individual actions if needed. The integration aims to enhance functionality and invites feedback to ensure a smooth user experience.

The blog post discusses the challenges and solutions associated with using large language models (LLMs) to generate SQL queries, a process that can enhance interactions with SQL databases by allowing natural language inputs. While LLMs can write SQL, they often hallucinate, creating invalid or nonexistent tables and fields, and struggle with limited context windows due to the vastness of SQL databases. Solutions include grounding LLMs in reality by providing detailed database schema descriptions and example data, limiting output size to manage context window constraints, and using error correction techniques similar to those employed by human data analysts. The post also highlights best practices such as few-shot learning and using subqueries to improve SQL generation accuracy, urging the community to share insights and engage in further development through platforms like Discord. A webinar is announced to discuss these insights, emphasizing the community-driven approach to refining LLM-SQL interactions.

The introduction of a PromptSelector concept aims to address the challenges faced with default prompt templates not working uniformly across different models, particularly highlighted after OpenAI's release of a new ChatGPT API. This API required new abstractions and led to issues with old prompt templates, which performed inconsistently in chat settings compared to GPT-3. The PromptSelector allows for the automatic selection of an appropriate PromptTemplate based on the model being used, offering a more seamless "Just Works" functionality. This approach is designed to accommodate different prompt requirements for various models, including ChatModels and traditional models, as well as different model providers and versions. The transition to using PromptSelectors will occur gradually, starting with popular chains, and community feedback is encouraged to enhance this development.

Origin is an innovative app developed by UC Berkeley students to address the common issue of browser tab clutter by organizing browsing history into context-aware workspaces and providing automatically generated summaries. Designed during Stanford's TreeHacks, Origin uses LangChain's summarization and chat capabilities, along with traditional machine learning techniques like K-Means and collaborative filtering, to enhance user interaction with saved web information. It creates semantic search, recommendation systems, and chatbots specific to each workspace, facilitating knowledge centralization and easy access. However, the team acknowledges challenges such as high compute costs with GPT-3 models and the need for further development to refine the summarization and other features for practical use. They are committed to addressing these engineering and design challenges to transform the hackathon project into a viable product, leveraging LangChain as a crucial tool in this endeavor.

OpenAI's release of a new ChatGPT endpoint introduces a distinct API format that significantly differs from the existing GPT-3 API, shifting from a text-based input/output interface to a message-based structure comprising "user", "assistant", and "system" messages. This new format necessitates revisions in LangChain's abstractions to accommodate these changes, allowing for interoperability with both current and potential future chat-based models. The differentiated message types aim to enhance model safety by structuring inputs to prevent issues like prompt injection attacks, although practical effectiveness remains uncertain. LangChain's adaptation efforts are focused on creating versatile abstractions that are not only OpenAI-specific but also applicable to other chat-style models anticipated from companies like Anthropic and Google. These efforts include developing chat message templates and the concept of a PromptValue for seamless interoperability between traditional and chat-based LLM APIs. The overarching goal is to enable users to fully leverage the capabilities of chat models while ensuring compatibility with various language model providers.

OpenAI recently unveiled a new ChatGPT API, prompting comparisons with the existing GPT-3 API in question-answering tasks. The evaluation focused on the "generation" step of a question-answering chain, using the text-davinci-003 model and ChatGPT with VectorDBQAChain prompts. The ChatGPT API distinguishes itself by using a list of messages with distinct roles—user, system, and assistant—compared to the single string input of GPT-3. In testing, ChatGPT and GPT-3 were evaluated for their ability to answer questions generated from a dataset, with results showing slight differences in grading between the two models. GPT-3 was noted for being succinct, while ChatGPT provided more detailed and polite responses, though the evaluation was limited by a small sample size. The study highlighted the need for further exploration of the ChatGPT API's role parameter and its optimal use, as well as the development of suitable abstractions for this new API paradigm.

Agent toolkits represent a new abstraction designed to enable developers to create agents tailored for specific use cases, such as interacting with relational databases or OpenAPI specifications, and are supported in both Python and TypeScript. These agents leverage a system that uses a language model to decide actions iteratively, enabling them to combine external knowledge and computation, perform iterative planning, and manage errors effectively. Some examples of these toolkits include the SQLDatabaseToolkit, which allows an agent to interact robustly with databases, and the OpenAPIToolkit, which enables an agent to make accurate API requests based on OpenAPI specifications. Additional examples include agents that interact with JSON data, vector stores, and data formats like Pandas DataFrames and CSVs, illustrating the versatility and potential applications of this approach. The development of agent toolkits aims to expand the range of possibilities for interacting with various tools and utilities, with plans to introduce more toolkits in the future.