Ray Summit 2022 Stories - Large Language Models

Ray Summit 2022 highlights the use of Ray in training large language models, including OpenAI's GPT3, by companies such as Cohere.ai and UC Berkeley's Alpa project. Ray is used to automate model parallel training and serving, simplifying the coordination of distributing tasks across multiple hosts. The platform is also scalable, allowing for efficient training of large models like GPT3. Additionally, Ray is not just limited to language models, but can be used for generic distributed tasks as well. Companies such as Cohere.ai are using Ray on top of TPUs to distribute and schedule different tasks across many TPU hosts, while UC Berkeley's Alpa project uses Ray to automate model parallel training and serving of large language models like GPT3.