Company
Date Published
Author
Marco Palladino
Word count
794
Language
English
Hacker News points
None

Summary

Insomnia has introduced the AI Runner with the release of Insomnia 10, a managed SaaS service designed to enhance and secure LLM traffic for developers' applications. This new tool accelerates LLM traffic up to 20 times by utilizing semantic caching, while also providing out-of-the-box AI guardrails for security. The AI Runner allows developers to use multiple LLMs through a single OpenAI-compatible interface, making it easier to build faster and more secure AI-driven user experiences. Developers can create multiple AI Runners, each with its own configuration, and quickly integrate them into existing applications by changing just one line of code. The service offers flexibility with settings like similarity thresholds and caching time-to-live (TTL) and supports storing credentials within the AI Runner, eliminating the need for frequent application updates. Powered by Kong's AI Gateway technology and running on Kong Konnect's enterprise infrastructure, the AI Runner is available for free to all Insomnia users, with options to self-host for additional features.