dlt+ClickHouse+Rill: Multi-Cloud Cost Analytics, Cloud-Ready

In Part 2 of the FinOps project, the focus shifted from local setups to a cloud-native approach using ClickHouse Cloud and Rill Cloud for analyzing and sharing cloud cost data. By utilizing GitHub actions for automation, the project extracts data daily from AWS, GCP, and Stripe, anonymizes it, and demonstrates the process of moving data from local Parquet files to ClickHouse Cloud for enhanced scalability. The integration of Rill Cloud enables the visualization and deployment of dashboards for cost analysis, illustrating the project's capacity to act as a template for FinOps by combining financial management with cloud operations. The article also discusses various challenges and solutions encountered during the transition, such as YAML and Python configurations, data anonymization, and the importance of a declarative data stack for ease of automation and configuration. This comprehensive setup provides a robust foundation for organizations to gain insights into their cloud spending while maintaining flexibility and scalability in their data analysis processes.