LangSmith LLM Gateway: runtime controls for production agents

LangSmith LLM Gateway is now available in public beta, offering a centralized governance layer for managing runtime controls over model calls in production environments to prevent issues like cost overruns and outages. Designed to provide essential controls such as spend caps, rate limits, model fallbacks, and sensitive data handling, the Gateway facilitates consistent governance across agents, models, and providers, thus avoiding vendor lock-in. By allowing engineering teams to define policies once and enforce them uniformly, it mitigates the risks of costly mistakes and outages that can occur when model providers face operational issues. The Gateway supports popular providers and custom models, providing a single interface for routing calls between agents and model providers, and it offers hosted inference for open models through Fireworks to enhance performance and control. Teams have already adopted the LLM Gateway to implement hard spend caps and centralized billing, demonstrating its utility in scaling AI agents effectively. As the Gateway evolves, future enhancements will include broader risk coverage and CI/CD controls to further streamline production model governance.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.