October 2024 Summaries

Ngrok has introduced two new primitives, internal endpoints, and the forward-internal action, to enhance its traffic routing capabilities, enabling users to create flexible routing topologies based on various request attributes like headers, paths, and subdomains. Internal endpoints are secure, non-public endpoints that facilitate routing traffic through the forward-internal action, allowing users to manage traffic flows without exposing services to the public internet. This advancement addresses the challenges of traffic orchestration by allowing path-based, domain-based, header-based, and geolocation-based routing, which can be configured through ngrok's Traffic Policy module. These features are particularly beneficial for use cases such as self-service app delivery, centralized security management for developer environments, on-demand public addressability, and live traffic debugging. The new primitives are aimed at providing more security and flexibility in traffic management, and they are available through ngrok's pay-as-you-go plan, with detailed guidance provided in their developer documentation.

Ngrok has introduced Traffic Policy, a feature that enhances the flexibility and control developers have over the traffic to and from their applications and APIs, using the Common Expression Language (CEL) for detailed traffic manipulation. This innovation addresses previous challenges with ngrok's module system, which required cumbersome configurations and lacked the ability to conditionally apply rules based on traffic attributes like paths or headers. Traffic Policy allows for programmable expressions to be applied at different phases of traffic handling, such as when a connection is established or during HTTP requests and responses, enabling more nuanced and efficient traffic management. The new system also simplifies the process of updating and deploying traffic policies without extensive changes to SDKs or applications, as it moves away from rigid structures to more flexible string-based configurations. As ngrok continues to integrate old module functionalities into Traffic Policy, they are also exploring new capabilities such as action chaining, specific service routing, and global security policy configurations, all aimed at making it easier and safer for developers to bring their services online without deep networking expertise. With Traffic Policy now generally available, ngrok invites users to explore its potential, offering free initial requests and extensive documentation to support developers in shaping their traffic effectively.

You can accomplish a wide range of tasks by manipulating headers in the requests and responses that pass through your API gateway. For example, you can update the host header to avoid errors when making requests to upstream services, remove service details from response headers to protect sensitive information, add tracing/logging details to inject crucial information about the request-response lifecycle into your upstream service, streamline headers to control logging compute and costs, create SLA rules and report on metrics, enable geo-aware services by adding a header that triggers a localization service based on country codes, and take further action on added, edited, or removed headers. These actions can be combined using expressions and CEL interpolations to achieve more complex logic and processing at the API gateway level.

Rate limiting can transform an API in nuanced ways beyond just blocking potential floods of incoming requests. It can improve availability, UX, fairness, and internal development velocity when implemented with the nuance of a natural rivershed rather than a binary dam-like approach. This involves creating sophisticated security policies that account for variables like authentication state, request values, location, and more to protect against advanced attacks and fraudulent usage. Nuanced protection against accidents can be achieved by implementing catch-all rate-limiting policies for general protection, as well as subtle limits to protect from mistakes from known entities. Rate limiting can also lead to performance gains when deployed correctly, with the ability to claw back latency based on how the API gateway is designed and where its algorithms are run. Additionally, nuanced pricing and go-to-market strategies can be enabled through usage-based billing, while UX improvements can be achieved through nuanced rate-limiting policies that filter requests based on user behavior, location, and other factors. In multitenant systems, fairness requires sophisticated rate-limiting policies that account for every possible shred of data attached to requests. Finally, a developer-friendly implementation of rate limiting can improve the internal developer experience by providing flexibility, familiarity, environmental flexibility, GitOps-readiness, and centralization. By layering in nuance into rate-limiting practices, businesses can enrich their API's ecosystem for both their business and every user downstream.

Ngrok, initially designed to bring local services online with ease, has evolved into a versatile tool used by 80% of the Cloud 100 as a universal gateway for API and service development. A recent example demonstrates how to set up secure WebSockets with ngrok and the Twilio Media Stream API for live audio transcription and summarization using OpenAI's Whisper and ChatGPT models. The process involves an Express server handling POST requests from Twilio, which streams audio that is saved, transcribed, and summarized, showcasing ngrok's capability to securely expose local services to public APIs without configuration changes. This workflow underscores ngrok's value in speeding up development by eliminating the need for pushing updates to production systems, illustrating its foundational role in testing and tunneling local services to public access, which remains crucial for developers.