Building Kepler

Astronomer developed Kepler, a Slackbot and CLI tool, to address the growing need for scalable access to data and contextual insights across different departments like product management, sales, finance, and marketing. Traditional data team operations often involved handling ad hoc requests and providing guidance, which became unsustainable as the company expanded. Kepler was designed to empower individuals to explore data independently by leveraging a rich context layer built on existing warehouse metadata and code, rather than relying on a complex semantic layer. This context layer incorporates schema metadata, enrichments like table popularity and comments, and code to enable efficient semantic searches and informed SQL query generation. Kepler supports iterative analysis and complex data manipulations via a persistent Jupyter kernel, while playbooks capture successful analysis patterns for repeatability and trust-building. The tool has transformed data interactions at Astronomer, encouraging more users to engage with data and deepening conversations with the data team. By open-sourcing the core components, Astronomer aims to help other companies build similar tools, with ongoing enhancements planned to improve playbook management and automate insights delivery.