The Last Mile is Solved by Slop

In the blog post "The Last Mile is Solved by Slop," Adrian Brudaru, Co-Founder and CDO, discusses an innovative approach to simplifying the data modeling process in business intelligence (BI) applications by using a prototype workflow called "slop." This method leverages a Local Language Model (LLM) wrapped in a scaffold to automate the traditionally manual task of creating a star schema from raw data, significantly reducing the time required from days to minutes. The process involves several steps, including schema inference, question formulation, and model validation, using a mix of code, templates, and prompts that guide the LLM through a disciplined sequence of actions. This approach democratizes data engineering skills, allowing less experienced users to effectively perform tasks that traditionally required senior-level expertise. By automating the transformation of raw data into a structured format ready for BI tools, the prototype showcases the potential for LLMs to redefine the role of data engineers, shifting the focus from technical implementation to strategic decision-making and outcome definition.