Building Ghostwriter Chat

At Replit, they developed Ghostwriter Chat, a chat application that uses Large Language Models (LLMs) to help users with programming questions within their IDE. The technology powers the ability for Ghostwriter to access and answer questions about a user's program without copying and pasting entire code blocks. To overcome latency issues, streaming results is used instead of waiting for completion. A carefully constructed prompt with sufficient context can yield detailed responses, while simple prompts may not be helpful. To support different models and APIs, the system takes varying information from different sources, chooses what to include and exclude, and converts it to various prompt formats. The Ghostwriter Debugger works by supporting the many ways programs display and throw errors, and heuristics are used to get around token limits on LLMs, which limit the amount of context that can be included in a chat conversation. By assuming the most important files for Ghostwriter to know about are the files being currently worked on, it enables Ghostwriter to truly act as a pair programmer. As AI and LLMs become stronger, future versions of Ghostwriter will only get smarter, better, and faster, unlocking more seamless interaction with IDEs and leveraging more file context.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.