December 2025 Summaries

Prompt caching is a strategy that significantly reduces the cost of using large language model (LLM) tokens by tenfold. This approach involves storing and reusing previously generated prompts to minimize the number of tokens needed for future queries, thereby optimizing resource usage and reducing expenses. Sam Rose, a Senior Developer Educator at ngrok, explores this concept in detail, offering insights into how developers can implement prompt caching to maximize efficiency and cost-effectiveness when working with LLMs.

Nijiko Yonskai, a Principal Product Manager with experience at Kong and Postman, introduces the ngrok AI gateway, which is currently in early access. This new gateway aims to enhance the user experience by integrating AI capabilities into ngrok's platform. The announcement highlights the potential for AI to streamline processes and improve functionality for users, with the early access phase allowing for feedback and iterative development. The introduction of this AI gateway is part of ngrok's ongoing efforts to innovate and provide advanced solutions to its user base, leveraging Yonskai's expertise in product management and user experience design.

Ben Chan, a software engineer at ngrok, details the process of building ngrok's Web Application Firewall (WAF) by utilizing the OWASP Core Rule Set (CRS) and Coraza. This endeavor involved implementing a robust security framework to protect against common web vulnerabilities. By leveraging the OWASP CRS, which is a set of generic attack detection rules, and integrating it with Coraza, an open-source WAF engine, ngrok enhanced its security measures to safeguard web applications effectively. The approach underscores the importance of employing tested and community-supported security tools to maintain a resilient and secure web infrastructure.