October 2023 Summaries

GPT-4 is OpenAI's latest model announced in March 2023, which shows impressive performance on various tasks including professional medical and law exams, expanding the maximum input length to 32,768 tokens, but its architecture and training datasets remain unclear. GPT-4 wins our pick for a fully hosted, API-based LLM due to its strong track record of OpenAI, although a subscription to ChatGPT Plus is required for access. ChatGPT is another text-only model released by Open AI in November 2022, designed to engage in natural language conversations, with basic access available without a subscription and suitable for personal projects or experimentation. LLaMA 2, released in July 2023, is Meta AI's next-generation open-source language understanding model, which comes in various sizes and variants for code understanding and completion, fine-tuning for commercial and research purposes, and has double the context length of its predecessor. The FALCON series of models developed by the UAE's Technology Innovation Institute shows impressive performance on pre-trained Open Large Language Models and is available for both research and commercial use. Mistral 7B, announced in September 2023, outperforms Llama2 on many benchmarks with a relatively small size that doesn't require monstrous GPUs to host, making it our pick for the best overall self-hosted model for commercial and research purposes. GPT-3 is OpenAI's pre-trained model fine-tuned on a particular task, exhibiting impressive few-shot and zero-shot performance on NLP tasks. BLOOM released in November 2022, is a multilingual LLM that generates text in 46 natural languages and 13 programming languages, with the aim of developing a more transparent and interpretable model. LaMDA announced in May 2021, is a model designed to have more natural and engaging conversations with users, built on an earlier Google Chatbot called Meena. MT-NLG uses the architecture of the transformer-based Megatron to generate coherent and contextually relevant text for various tasks, available via API. LLaMA announced February 2023 by Meta AI, is a model available in multiple parameter sizes from 7 billion to 65 billion parameters, with access only available to researchers, government affiliates, those in academia, and after submitting an application to Meta. Stanford Alpaca was announced in March 2023, fine-tuned from Meta's LLaMA 7B model and trained on 52k instruction-following demonstrations, aiming to help the academic community engage with models by providing an open-source model rivaling OpenAI's GPT-3.5 models. FLAN UL2 is an encoder decoder model souped-up version of the T5 model trained using Flan, exceeding prior versions' performance and available for self-hosting or fine-tuning. GATO announced May 2022, deepmind's multimodal model capable of working on not just text but other modalities and performing multiple tasks such as image captioning and controlling a robotic arm, although its release remains unclear. PaLM, announced April 2022, is based on Google's Pathways AI architecture aiming to build models that can handle many different tasks and learn new ones quickly, achieving state-of-the-art performance on many language-related tasks. Claude is described as a "next generation AI assistant" by Anthropic, available in two modes: Claude and Claude Instant, with limited details about its training process or model architecture. ChatGLM announced March 2023 by Tsinghua University's Knowledge Engineering Group, is a bilingual Chinese-English language model available for download at HuggingFace, optimized for the Chinese language and with an Apache-2.0 license allowing commercial use.

The blog post discusses the rapid evolution of the language modeling space since the introduction of the transformer architecture in 2017 and highlights various notable large language models (LLMs) from different organizations. Models like GPT-4 and ChatGPT from OpenAI, LLaMA from Meta AI, and FALCON from the Technology Innovation Institute, among others, are examined for their unique features, sizes, and use cases. The post emphasizes the importance of instruction-following LLMs, which differ from task-specific models, in performing a wide range of tasks based on user instructions. The article also touches on Vectara's role in utilizing LLMs to enhance search experiences by providing semantic and contextually relevant results, ultimately aiming to transform how users interact with large volumes of business data.

Creating an effective Retrieval-Augmented Generation (RAG) pipeline that provides good responses in multiple languages can be more complicated than it initially appears, requiring a good chunking strategy, a state-of-the-art embedding model, and proper implementation. The choice of embedding model significantly impacts RAG performance, with Vectara's new Boomerang model outperforming OpenAI and Cohere models in some cases, especially in non-English languages like Hebrew and Turkish, where it retrieves relevant information from the data more effectively than its competitors. By using Boomerang integrated into Vectara's "RAG as a service" architecture, users can build effective GenAI applications with improved performance across multiple languages.

The blog post explores the significance of embedding models in retrieval-augmented-generation (RAG) pipelines, highlighting Vectara's new Boomerang model and its advantages over existing models like those from OpenAI and Cohere. It begins by discussing the role of text chunking and embedding models in semantic search, explaining how these models convert text into vectors to facilitate accurate retrieval of information. Vectara's Boomerang model is presented as a superior option, especially in multi-lingual contexts, demonstrating notable performance improvements in languages like Hebrew and Turkish compared to its competitors. The post includes a practical demonstration using a RAG pipeline for question-answering based on the LLAMA2 paper, showcasing Boomerang's efficacy across different languages and emphasizing the importance of a well-structured RAG setup. It also shares a success story from SonoSim, illustrating how Vectara's AI solutions enhanced their search capabilities and training platform efficiency.

We evaluated the performance of Vectara's new AI-powered search engine, Boomerang, against a popular question/answering model, USE-QA. We tested Boomerang on five datasets for different applications, including e-commerce search and scientific search, and found that it provided superior performance in four of the five datasets. The evaluation also highlighted areas where cross-language information retrieval could be improved. We suggested integrating Quepid with Vectara to test and tune its engine further, which is now live on www.quepid.com. Additionally, we provided feedback on available features and suggested upgrades, such as an "upsert" capability and more useful metadata filters. The Vectara team is using our results to improve Boomerang, demonstrating their commitment to testing and evaluation.

The rise in AI technology has disrupted the search engine landscape, offering a plethora of new and traditional options for users and businesses, necessitating the development of robust metrics to gauge the quality of AI-powered search engines. OpenSource Connections (OSC), with a strong background in search measurement, partnered with Vectara to evaluate its Boomerang AI-powered search model against the popular Universal Sentence Encoder QA (USE-QA), finding Boomerang superior in most tested datasets. The evaluation highlighted areas for improvement, especially in cross-language information retrieval and vague query handling. OSC also developed a Quepid integration to further enhance Vectara's search capabilities, leading to feature upgrades and bug fixes. The collaboration with Vectara, marked by transparency in sharing test results, aims to contribute to the discourse on search quality measurement and improve search engine technologies, with practical applications demonstrated in SonoSim’s advanced search interface for medical training.

Hybrid search has emerged as a powerful approach to improve in-app product search, allowing developers to combine the strengths of semantic and lexical search techniques. By leveraging large language models (LLMs) and keyword-based retrieval, hybrid search systems can deliver more accurate search results, intuitive user interaction, and personalized engagement. This approach also supports cross-language searches and is critical for improving the experience for non-English-speaking users. Implementing hybrid search can lead to increased user loyalty, reduced onboarding friction and costs, improved customer satisfaction, better conversion rates, and new revenue streams. Hybrid search platforms like Vectara provide an end-to-end solution for developers, offering a simple-to-use API and comprehensive approach to search that yields the most comprehensive results.

Exceeding user expectations and minimizing friction are crucial for successful application experiences, and Hybrid Search, which combines keyword and semantic search methods, significantly enhances this by improving search accuracy, user interaction, and personalization. Traditional keyword search systems often struggle with understanding intent and synonyms, whereas hybrid search leverages large language models (LLMs) to better grasp the context and intent of queries, offering superior cross-language support. By integrating both approaches, hybrid search ensures more accurate and personalized search results, leading to increased user satisfaction, loyalty, and conversion rates. Vectara, a platform offering hybrid search capabilities, allows users to fine-tune search settings for optimal results, providing a robust tool for developers to enhance product experiences. Through the use of hybrid search, companies can improve user engagement and reduce churn, ultimately benefiting their bottom line by offering a competitive advantage in the market.