How we built Insights AI with Inngest

In an effort to improve the usability and adoption rate of its SQL-based query tool, the company developed Insights AI by integrating AI to allow natural language queries, thus addressing the steep learning curve associated with understanding SQL and specific schema knowledge. The transition involved shifting from OpenAI's models to Anthropic's Claude for better context handling and instruction adherence, significantly enhancing query accuracy and response times. They also implemented checkpointing to drastically reduce latency from 15-30 seconds to 3-6 seconds per query and refined their prompt engineering, moving from vague system prompts to explicit, detailed instructions and examples, resulting in an improved success rate of 90% for queries. The development of Insights AI demonstrated the company's capability in building AI-powered features, increasing user engagement by making data querying accessible to non-technical users, and reducing frustration for those encountering errors in custom queries. The project highlighted the importance of prompt engineering, model selection, and infrastructure management, serving as a successful example of how dogfooding can refine a platform's capabilities for both internal and customer benefit.