January 2024 Summaries

JSON mode and function calling are two new features introduced in the Together Inference API, designed to provide developers with more flexibility and control over their interactions with LLMs. JSON mode allows specifying a JSON schema for the output of the model, resulting in structured and predictable responses, while function calling enables intelligent output of JSON objects containing arguments for external functions defined by the developer. This feature is particularly useful when real-time data access or custom logic is required. The API supports these features in Mixtral, Mistral, and CodeLlama models with more coming soon. The introduction of these features aims to enhance the versatility and functionality of AI applications built on the Together AI platform.

The Together Embeddings endpoint offers higher accuracy, longer context, and lower cost than other popular platforms, with 8 leading embedding models available, including top-performing models from the MTEB leaderboard. The endpoint also supports state-of-the-art long-context M2-Retrieval models up to 32k context length, making it suitable for applications such as retrieval augmented generation (RAG), which aims to overcome limitations of generative AI models by finding relevant information from a given knowledge base through embeddings and providing the information to a generative model. The endpoint is fully OpenAI compatible, allowing developers to easily switch between platforms, and offers competitive pricing, with some models up to 4x cheaper than others. It also provides integrations with popular frameworks such as MongoDB, LangChain, and LlamaIndex for RAG, making it an attractive option for building successful AI applications.

Together AI has launched the Together Embeddings endpoint, enabling users to build their own powerful RAG-based applications directly from the platform using Langchain. RAG (Retrieval Augmented Generation) combines generative models and retrieval models for knowledge-intensive tasks, improving performance and accuracy by leveraging external data sources during response generation. Building a RAG system can be cost and data efficient without requiring technical expertise to train a model, and fine-tuning an embedding or generative model can further improve the quality of the solution. The process involves creating a vector store using an embedding model, retrieving relevant data examples, augmenting the information with a prompt, and obtaining the final output from a generative model. An example demonstrates how to incorporate recent knowledge into a RAG application using the Together API and Langchain, providing accurate and up-to-date responses compared to relying on pre-trained models.

The text discusses the development of long-context retrieval models using Monarch Mixer, a recent model family that aims to improve the scaling properties of Transformers along two axes – sequence length and model dimension. The authors release a preview of several models, including long-context versions of M2-BERT up to 32K context length and embedding versions fine-tuned for long-context retrieval. They also introduce a new benchmark called LoCo, which is designed to evaluate the performance of long-context retrieval models on tasks with long documents. The authors report promising results, demonstrating that their long-context M2-BERT models can outperform much larger models on this benchmark, suggesting that long-context models are beneficial for retrieval. The authors also discuss challenges in training long-context models, including adapting the BERT pretraining pipeline and fine-tuning the model using a suitable loss function. They propose a new loss function called orthogonal loss, which pushes the cosine similarity of positive pairs to 1 and the cosine similarity of negative pairs to 0.

You can build a powerful Retrieval Augmented Generation (RAG) application using Together AI's cloud platform and LlamaIndex, which provides fast and cost-efficient training without requiring technical expertise to train a model. This approach leverages both generative models and retrieval models to improve knowledge-intensive tasks by providing up-to-date information from external data sources during response generation. By creating a vector store and indexing source documents using an embedding model of your choice, you can retrieve relevant information, augment it with the original query, and use a large language model (LLM) to generate accurate responses. This approach has been demonstrated through a quickstart example that incorporates a new article into a RAG application using the Together API and LlamaIndex. The tools provide numerous advantages, including faster training, lower costs, and improved performance, making it an attractive option for building innovative solutions.

Together AI provides a fast cloud platform for building and running generative AI applications, including the launch of its Together Embeddings endpoint, which allows users to build powerful RAG-based applications using MongoDB's Atlas Vector Search. RAG combines generative models with retrieval models to improve performance and accuracy in knowledge-intensive tasks. To use RAG, users populate a vector database using an embedding model, retrieve relevant data examples, augment the retrieved information, and obtain the final output from a generative model. The Together AI platform provides a step-by-step guide on how to implement RAG with MongoDB Atlas, including setting up an account, creating an embedding function, storing embeddings, creating a vector search index, retrieving data, augmenting and generating outputs, and fine-tuning models. This tutorial demonstrates how to build a RAG application using the Airbnb listing review dataset and showcases the potential of combining generative AI with semantic search for high-quality applications.