Company
Date Published
Author
Tamás Srancsik
Word count
594
Language
English
Hacker News points
None

Summary

We are building a package that collects information about Bitrise's GitHub repositories into a regular relational database for ad-hoc reports. We need to process the data from the GitHub API, which involves GET requests to retrieve repository and pull request information. To handle JSON data, we use Xplenty's ETL tasks, including Curl functions to extract data from the API responses, Select components to parse the data, and Flatten() to generate rows for each element of a bag. The final step is parsing the JSON key-value pairs into new fields and values, which will be stored in a relational database with Post-action SQL commands creating indexes on the table.