August 2024 Summaries

You're Not an Engineer. Why Should You Care About RAG?` is about Retrieval Augmented Generation (RAG), a technology that provides human-like responses in a matter of seconds by harnessing the power of deep learning tools. RAG offers a two-step approach, first retrieving relevant data from various sources and then generating accurate answers to prevent hallucinations. The author, Miluska Berta, emphasizes the importance of having a reliable RAG solution to ensure efficiency and accuracy in AI-powered applications. She also highlights the limitations of GenAI tools and the need for a trusted GPS tool like RAG to navigate complex data landscapes. By adopting RAG as a service, organizations can work smarter, not harder, and achieve better results in their AI endeavors.

The author recounts their journey to joining Vectara, a company co-founded by Amr Awadallah, highlighting the appeal of being part of a revolutionary AI project and the successful Series A funding that facilitated their recruitment. At Vectara, the author explores the complexities of the AI landscape, particularly focusing on Generative AI (GenAI) and Retrieval Augmented Generation (RAG) as a service. RAG is described as a two-step approach that enhances GenAI tools by retrieving relevant data to provide accurate, up-to-date answers and prevent hallucinations, or incorrect responses. The author argues that while understanding RAG is important, the real focus should be on the results it delivers, likening it to a GPS that efficiently guides users with accurate information. Conversations with AI conference attendees reveal challenges faced by those using GenAI tools, with Vectara's RAG offering a compliant solution for processing both structured and unstructured data. The piece concludes with an encouragement to leverage Vectara's platform for efficient and accurate AI solutions.

Vectara Portal is a no-code environment for building chat applications with documents, powered by Vectara's trusted RAG-as-a-service platform. It enables users to create AI assistants without coding, providing higher accuracy of responses with reduced hallucinations. Users can sign up for a Vectara account and set up their profile before creating a portal, which is a single application connected to the RAG-as-a-service that can be customized with only a single configuration parameter. Portals can be created based on uploaded documents, such as PDFs, and provide features like chat interfaces, search functionality, and reference links. The Portal app is hosted by Vectara and available immediately at https://portal.demo.vectara.com, while the source code is made available under the Apache 2.0 license for open-source use.

Vectara Portal is a beta service designed to enable non-developers to create generative AI applications without requiring coding skills, utilizing Vectara's RAG-as-a-service platform. This service allows users to build AI assistants that can chat with, search, or summarize data through a simple interface. To use Vectara Portal, users need to sign up for an account and configure their profile with Vectara credentials. The portal offers options for creating applications based on semantic search, question-answering, or chat functions, using documents uploaded by the user. It also includes features for sharing portals publicly or keeping them private, and the source code is available as open-source under the Apache 2.0 license. Vectara encourages feedback and contributions to improve the platform, aiming to make AI accessible to a broader audience.

Vectara has introduced user-defined sorting functions, allowing users to override the ranking function with their own custom functions. These new capabilities provide more granular control over result ordering and reduce LLM costs by considering additional signals such as recency of information, price, or average rating. Users can use mathematical functions and metadata fields to influence the ranking, enabling them to customize Vectara's functionality to meet specific application needs. The new UDF (user-defined function) reranking capabilities are available for use in various scenarios, including retrieval-augmented generation and summarization tasks, allowing users to tailor the information fed into the generative LLM.

Vectara has introduced user-defined sorting functions, allowing users to customize the ranking of search results based on factors beyond textual relevance, such as recency, price, or product ratings. This functionality is particularly beneficial for integrating legacy search applications and offers greater control over search result ordering while potentially reducing costs associated with large language models (LLMs). Users can override default ranking functions by providing custom functions that access document metadata and Vectara relevance scores, enabling sorting by various criteria like price or review scores. These reranking capabilities occur before results are sent for retrieval-augmented generation, allowing users to influence the information considered by generative models and avoid outdated data. Vectara encourages users to explore these new capabilities and share feedback through their community platforms.

Vectara-Agentic is a Python package that enables the creation of AI Assistants and AI Agents using the Agentic RAG (Reasoning and Augmented Generation) methodology, which leverages Large Language Models (LLMs). It abstracts away many details and provides functional flexibility, a broad set of tools out of the box, and ties back to the scalable and secure enterprise RAG platform from Vectara. By leveraging tool use, LLM reasoning processes can be emulated within the AI Assistant or Agent, allowing them to safely act on behalf of users to achieve desired outcomes such as sending emails or booking flights. The package is designed to make building Agentic RAG applications easy and straightforward, with a focus on reducing hallucinations and improving trust in the output. It supports various LLM providers and provides examples of how to use it for different use cases, including a legal assistant that helps with case law research.

Autonomous software agents have been a topic of interest since the mid-20th century, evolving from simple query-response systems to complex AI Assistants and AI Agents enabled by Large Language Models (LLMs). The introduction of Retrieval Augmented Generation (RAG) has enhanced these agents by allowing them to access relevant, accurate information, reducing errors and building user trust. Agentic RAG represents the latest advancement, empowering AI Assistants to not only respond to queries but also execute tasks like booking flights or sending emails by interacting with external tools. The beta release of vectara-agentic, a Python package, aims to simplify the creation of such AI applications, leveraging its integration with the LlamaIndex open-source package. This allows developers to define tools and instructions specific to their use case, ensuring the AI's actions align with user needs. Vectara-agentic's support for industry-specific tools and its ability to interface with various LLMs highlight its potential in diverse applications, from legal research to financial assistance, promising a future where AI can perform complex tasks autonomously while minimizing errors.

HHEM-2.1 is an improved version of the previous model HHEM-2.0, which outperforms both GPT-3.5-Turbo and GPT-4 for hallucination detection in three languages: English, French, and German. The new model has been integrated into Vectara's RAG-as-a-service platform and is automatically included with every call to the Query API, making it easy for enterprise developers to build trusted GenAI applications. HHEM-2.1 also offers a more accurate hallucination detection performance compared to its predecessors, with a better recall and precision in identifying hallucinations where they occur. The model has been benchmarked against other popular LLMs, including GPT-3.5-Turbo and GPT-4, and outperforms them in terms of F1 score, precision, and recall. Additionally, HHEM-2.1 is now available as an open-source model on Hugging Face and Kaggle, offering developers a more accessible option for building trusted GenAI applications. The new model also powers a revamped HHEM leaderboard that ranks LLMs based on their likelihood to hallucinate, providing a more accurate reflection of the true hallucination rate of LLMs.

HHEM-2.1, an advanced hallucination detection model from Vectara, marks a significant improvement over its predecessor, HHEM-2.0, by offering enhanced accuracy in detecting hallucinations across three languages—English, French, and German—without the latency and cost issues associated with the "LLM-as-a-judge" methodology. This model, integrated into Vectara's RAG-as-a-service platform, provides a Factual Consistency Score (FCS) that evaluates the trustworthiness of responses in real-time, making it well-suited for enterprise GenAI applications. Unlike previous methods that relied heavily on LLMs like GPT-4, HHEM-2.1 is a pure classification model that avoids the echo chamber effect and excels in both precision and recall. The model is open-source, available on platforms like Hugging Face and Kaggle, and performs efficiently on consumer-level GPUs, enhancing accessibility. HHEM-2.1 outperforms existing models in various benchmarks, offering a balanced precision/recall trade-off that bolsters its reliability. Additionally, Vectara has released a new LLM leaderboard powered by HHEM-2.1, which accurately ranks LLMs based on their propensity to hallucinate, reflecting HHEM-2.1's improved capabilities in handling longer sequences and delivering better precision and recall.