February 2025 Summaries

Cloud Endpoints is a new feature from ngrok that provides persistent, always-on endpoints, allowing users to route and respond to traffic even if upstream services go offline. Managed centrally via a dashboard and API, these endpoints employ ngrok's Traffic Policy system for seamless traffic handling similar to existing tunnels, enabling developers to create flexible architectures by using Cloud Endpoints as a "front door" to standardize traffic management. They offer several advantages over the deprecated Edges, such as simplified routing, fewer API calls, and integration with management automation tools like Kubernetes. This innovation allows for more intuitive traffic management, supporting various protocols and enhancing the ability to render custom error pages or redirect traffic when services are unavailable. Available on both free and pay-as-you-go plans, Cloud Endpoints aim to streamline operations by combining development autonomy and operational control, empowering developers to efficiently manage their service endpoints.

Adil Aijaz, CEO of Sam, discusses using ngrok to develop and manage the AI Sales Engineer and Call Intelligence platform, Sam. Sam functions as a virtual sales engineer, providing expertise during calls by capturing and processing Zoom meeting audio to detect and display customer questions in real-time using a distributed system architecture. The development process involves various interconnected services, making local development challenging. ngrok facilitates this by allowing developers to simulate production environments on their local machines, enabling them to test and iterate on components like transcription and question detection without disrupting live operations. This setup allows developers to safely test new prompts with production traffic, forwarding a percentage of questions to local endpoints and comparing results to ensure quality without affecting the live system. Overall, ngrok plays a crucial role in bridging the gap between local development and production, enhancing the ability to prototype and refine AI systems effectively.