The GPT-JT model was developed by Together as a decentralized, open-source AI fork of GPT-J-6B, fine-tuned on 3.53 billion tokens to outperform most 100B+ parameter models at classification tasks. It was trained with a new decentralized algorithm using slow internet connections and heterogeneous GPU hardware, resulting in improved performance and reduced communication overhead compared to traditional distributed learning algorithms. The model's strength lies in its ability to leverage community projects and datasets, such as EleutherAI's open models and Google Research's UL2 technique, to achieve state-of-the-art results. GPT-JT is now publicly available as open source, along with a live demo on the HuggingFace space, and offers a value chain that everyone can benefit from. The model's decentralized training approach paves the way for making AI more accessible to researchers and practitioners via decentralized computing, reducing costs and increasing efficiency.