At Replit, they've introduced Ghostwriter Chat, an AI-powered pair programmer that aims to provide developers with a powerful programming environment directly in their workspace. The goal is to give users the power they need without leaving their IDE, answering questions and debugging errors with the help of Large Language Models (LLMs). To overcome limitations such as token limits and request latency, Ghostwriter Chat uses streaming, generic prompt construction, and heuristics to balance context limits. It also includes features like the Ghostwriter Debugger, which proactively alerts users on errors and allows for single-click debugging. As AI and LLMs become more capable, Ghostwriter Chat will continue to improve, unlocking seamless interaction with IDEs, leveraging more relevant context, and enabling more complex actions.