June 2024 Summaries

GPT-4o from OpenAI is the top recommended large language model due to its performance, cost-effectiveness, and features including multimodality. GPT-3.5 Turbo is a close second for its high-quality chat interface and free tier options. CodeQwen-1.5 is an excellent choice for code understanding and completion tasks. Mistral 7B Instruct v0.3 is ideal for fine-tuning models for commercial or research purposes due to its permissive license and community support. Llama 3 70B offers high-quality performance as a self-hosted model, while Llama 3 8B provides an excellent balance of performance and hosting requirements for hobby projects. Gorilla OpenFunctions-v2 is the top choice for self-hosted models with tool use and function calling capabilities. These models are highly recommended based on their unique strengths and use cases.

The updated guide on selecting Large Language Models (LLMs) provides detailed recommendations for choosing the right model based on specific use cases, acknowledging the rapidly evolving LLM landscape with numerous new releases. It highlights several top picks, such as GPT-4o from OpenAI for its superior performance and multimodal capabilities, making it ideal for fully hosted, API-based applications. GPT-3.5 Turbo is recommended for those seeking a free, text-only model with a robust chat interface, while CodeQwen-1.5 by Alibaba is praised for its code understanding and completion capabilities. Mistral-7B-Instruct-v0.3 is favored for fine-tuning in commercial or research contexts due to its improved instruction-following features and permissive licensing. Meta's Llama 3 models, including the 70B and 8B variants, are highlighted for their high performance in self-hosting scenarios, with the 70B variant suitable for users with ample computing resources and the 8B variant ideal for those with limited budgets. Gorilla OpenFunctions-v2 is noted for its advanced function calling and tool use capabilities, making it a top choice for self-hosted models in agentic applications. The guide aims to simplify the selection process by offering tailored recommendations based on extensive testing and benchmarking.

RAGTime is an open-source bot developed by Vectara that enables users to interact with their Slack or Discord communities using a chatbot powered by the company's RAG platform. The bot listens to user messages and responds with relevant answers, providing features such as one-to-one chats, contextual conversations in threads, and personalized responses. To use RAGTime, users need to sign up for Vectara, create a corpus, ingest data into it, and follow step-by-step instructions to run the bot on their server and connect it to their Slack or Discord community. The bot is built using Python and SQLite, leveraging Slack Bolt and Discord.py to handle events, and uses Vectara's Query API to process natural language input and return precise responses.

RAGTime is an open-source bot designed for Slack and Discord that facilitates user interaction by responding to queries through Vectara's RAG platform. The bot, which can be integrated into servers, offers personalized one-to-one chats, the ability to ask questions by mentioning the bot, and maintains conversation context in threads. Built using Python and SQLite, RAGTime manages workflows and responds to direct messages or mentions by sending user queries to Vectara's Query API for precise answers. To implement RAGTime, users need to sign up for Vectara, create and ingest data into a corpus, and follow setup instructions for running the bot on a server. The developers encourage feedback for further improvements and provide a demonstration through their community Discord server, showcasing the bot's capability to answer questions based on Vectara's content.

Vectara's Boomerang is a leading embedding model launched in September 2023, designed for production use cases and balancing precision, embedding size, and storage costs effectively. It holds its own against the latest embeddings from OpenAI and Cohere, outperforming them on multilingual and cross-lingual performance while being comparable on English language performance. Boomerang is a small yet powerful model with high precision and low storage costs, meeting the criteria of achieving a small embedding size (<1K) with high precision and low storage costs. It stands out as the optimal choice for production use cases, effectively balancing quality, embedding size, and storage costs, making it a suitable option for users looking to leverage retrieval-augmented generation in their GenAI apps.

Vectara's Boomerang embedding model, launched in September 2023, is designed for production use cases and is compared to the latest embedding models from OpenAI and Cohere. Boomerang demonstrates comparable performance to OpenAI and Cohere models in English language tasks, while it significantly outperforms them in multilingual and cross-lingual tasks, according to benchmarks such as SQuAD, XQuAD-R, and MIRACLE. The model is designed to balance precision, embedding size, and storage costs, with Boomerang and Cohere V3/V3 light being optimal choices for achieving high precision with low storage costs. Boomerang's embedding size is 768, offering a cost-effective storage solution compared to larger models like OpenAI's text-embedding-3-large. Overall, Boomerang is presented as an effective choice for production environments, optimizing trade-offs between quality and resource efficiency.

Vectara has introduced version 2 of its API, which brings numerous quality-of-life improvements for software developers. The new API structure simplifies getting started with Vectara, allowing users to make simple API calls without complex JSON bodies. The authentication flow is also simplified, with standardized URLs and no need for special OAuth URLs. The API now follows a more RESTful design, using standard HTTP verbs and object-oriented URL structures. This makes it easier for developers to build applications on top of Vectara, with features such as standard error codes, better out-of-the-box defaults, and an open public specification. The company plans to release native language clients for popular programming languages in the future, further improving the developer experience.

Vectara has launched API v2, a new REST API designed to enhance the developer experience on their platform by simplifying the process of getting started and integrating more RESTful principles. This update introduces improvements such as a streamlined authentication flow, a standardized OAuth URL, and a more intuitive, resource-oriented API structure utilizing standard HTTP verbs for CRUD operations. API v2 also implements standard HTTP error codes, reducing the incidence of "silent" errors and providing better out-of-the-box defaults for configurations like hybrid search. These changes reflect feedback from developers and aim to refine the platform's usability while maintaining support for gRPC and ensuring compatibility with tools like Postman through integration with a public OpenAPI specification. Vectara plans to continue enhancing the platform by eventually offering native language clients for popular programming languages, further simplifying the development of GenAI applications.

Congratulations to the winners of the first-ever Built by Vectara Contest, which showcased impressive projects harnessing the power of generative AI and leveraging the Vectara platform. The three standout projects demonstrated functional utility and tackled pressing use cases such as integrating with smart glasses, public safety, calculators, and social media engagement. The contest highlighted the ease of use and speed of building RAG pipelines using Vectara's developer-friendly platform, making it accessible to users without deep machine learning experience. The winners' projects showcased the potential of generative AI to create over $200B in business opportunities for companies that leverage it correctly.

Generative AI presents significant business opportunities, potentially exceeding $200 billion, yet many projects remain in the prototype stage. To demonstrate the practical applications of generative AI, an initiative was launched inviting community members to use the Vectara GenAI platform to develop innovative projects. Participants had full access to Vectara's tools, including GPT-4-Turbo for tasks like generative summarization and custom prompts. Among the numerous submissions, three projects notably showcased the platform's capabilities: Wisdom Hoard, a Discord bot for managing and querying social media links; a Robert F. Kennedy Jr. chatbot that streamlines content retrieval and generation; and FirstAid RAG, an AI-powered virtual assistant offering accurate first aid guidance. These projects highlight the versatility and accessibility of Vectara's platform, emphasizing the ease of building robust RAG pipelines even for those without extensive machine learning expertise. The initiative underscores the transformative potential of generative AI across various industries and applications.