How we built Cloudflare's data platform and an AI agent on top of it

Cloudflare has developed two in-house tools, Town Lake and Skipper, to address the challenges of accessing and analyzing vast amounts of data generated across its global network. Town Lake serves as a unified data analytics platform that integrates various data sources into a single SQL interface, allowing for streamlined data access and analysis. Skipper, an AI data agent, provides a natural language interface for querying data, enabling users to obtain accurate and auditable answers quickly without requiring SQL knowledge. This system addresses the complexity of data sprawl, disparate systems, and the need for accurate, unsampled data while ensuring robust governance and security measures. By leveraging its own platform components, Cloudflare aims to empower employees with the ability to derive insights from data, fostering innovation and enhancing decision-making across the organization.