Inside Starburst’s hackathon: Into the great wide open(AI)

Starburst recently held its annual hackathon, Hack-a-Trino, where the team explored the integration of generative AI into data engineering processes, aligning with the current excitement around OpenAI. Three notable projects emerged: Automatic Data Classification, which leverages OpenAI’s LLM APIs to predict and tag data content for improved data management; Trino AI Functions, which uses ChatGPT and Hugging Face to translate natural language into complex SQL aggregations for advanced data analysis, including tasks like sentiment analysis and fraud detection; and No Code Querying with ChatGPT, designed to simplify data querying for business analysts by allowing natural language inputs, thus advancing Starburst's goal of data democratization. Additionally, a partner project with Aderas explored using Starburst and ChatGPT for cybersecurity analytics. Through these initiatives, Starburst gained insights into the rapid prototyping capabilities of AI models, recognizing their potential and limitations in generating accurate SQL queries, while also emphasizing the importance of data governance and security in cloud environments.