Battle against search and AI bots on your ngrok endpoints

You can configure ngrok's Traffic Policy module to block unwanted traffic from search engine bots and AI bots by using three options: blocking all bots with a `robots.txt` file, blocking specific bots by user agent, or restricting IPs with CIDRs. The first option prevents future bots from constantly pinging your service, while the second option ensures specific bots can't ignore your robots.txt and crawl your content. The third option blocks bots alongside other public internet traffic. Implementing all three options in sequence provides the utmost guarantee that your apps, services, and APIs are available to trusted peers.