May 2024 Summaries

Portkey has made significant advancements in May by introducing several features aimed at enhancing user control and integration with large language models (LLMs). Users can now manage organization and LLM spending through budget limits and rate controls, create API keys with specific permissions, and invite team members with tailored access levels. New integrations include Day 0 compatibility with major AI models from OpenAI and Google, as well as partnerships with providers like ZhipuAI, Predibase, and MonsterAPI. Portkey has also expanded its functionality by integrating with the Instructor library for structured output extraction and deepening its integration with Promptfoo to run evaluations across 200+ LLMs. Additional features include cache namespace simplification, Gemini function calling support, latency comparison tools, and the ability to deploy Portkey on one's own cloud infrastructure. Engaging with the community, Portkey's CTO participated in a Reddit AMA, and users are encouraged to join the Discord community for updates.

Portkey has partnered with F5, the creators of NGINX, to streamline the deployment, management, and monitoring of enterprise AI applications by integrating their AI Gateway and Observability Suite with F5 Distributed Cloud Services. This collaboration enhances intelligent large language model (LLM) orchestration, high availability, scalability, automated failover, resilience, comprehensive observability, and enterprise-grade security for AI-driven applications. The partnership leverages F5's robust application delivery and security features to address critical challenges in AI application deployment, ensuring seamless API request routing, workload distribution, and protection against cyber threats. By combining Portkey's expertise in AI application deployments with F5's traffic management and security capabilities, the joint solution offers businesses a powerful framework for delivering reliable, secure, and high-quality AI experiences.