August 2023 Summaries

The Llama-2-7B-32K-Instruct model achieves state-of-the-art performance for long-context tasks such as summarization and multi-document question answering while maintaining similar performance at a shorter context length compared to the base Llama-2-7B model. The model was fine-tuned using the Together API, which allows developers to easily build custom models with less than 200 lines of Python script. The fine-tuning process involves four main steps: distilling instructions from human inputs, training the model on a mixture of data sources, testing the model in the Together Playgrounds, and deploying it via the Together Inference API. The model outperforms other baseline models including GPT-3.5-Turbo-16k, Llama-2-7b-chat, Longchat-7b-16k and Longchat-7b-v1.5-32k on long-context benchmarks, demonstrating its robustness across these tasks. The model is now available for public use with the Together API, enabling developers to build custom models with ease.

To optimize inference, Together AI has simplified pricing for its cloud platform, allowing users to process a greater number of transactions per GPU, enabling better cost efficiency. The company has also released additional optimizations to speed up inference even more. With these updates, users can now run more efficient inference with the updated pricing, which includes lower costs and faster performance. This enables users to launch their own inference VMs for models they use, ensuring data privacy, while paying only for requests and an hourly hosting fee when launching their inference VM. The company offers a range of open-source AI models, including RedPajama, Llama 2, Falcon, and more, which can be used with the updated pricing. Together AI aims to provide users with more for less, enabling them to build and run fast AI models efficiently.