Doomed to Code: How we Teamed Up with Fireworks AI at MistralAI Hackathon to Conquer the Shores of Hell

At the Mistral AI Hackathon in San Francisco, a team comprising Bhav Ashok, Umut, and Paul developed a project using Fireworks AI to enable the Mistral-7B language model to play the video game Doom, a cult classic among hackers. The team aimed to create a "Game Copilot" by converting game visuals into ASCII representations, allowing the model to interpret and navigate the game's environment. Despite challenges like hardware constraints and a bug in their model's prompt, they successfully trained the model using Fireworks AI's fine-tuning API and rapid deployment capabilities. After overcoming obstacles such as simulating emergent behaviors and adjusting training data, the team was able to demonstrate an AI agent capable of autonomously playing Doom, ultimately winning first place in the hackathon's fine-tuning track. The project highlighted the potential of leveraging language models for complex tasks, with the Fireworks AI framework proving crucial to their success.