/plushcap/analysis/fivetran/can-ml-be-absorbed-by-the-dbms

Can ML be absorbed by the DBMS?

What's this blog post about?

The relationship between machine learning (ML) and database management systems (DBMS) is complex, with different strategies for integrating ML into DBMS depending on the composition of work. On one end of the spectrum are classic analytics teams using SQL data warehouses, where special SQL syntax for creating and evaluating ML models can be beneficial. BigQuery ML is an example of this approach. On the other end are data science teams primarily using machine learning libraries in Python, who may benefit from embedded DBMS like DuckDB. Many users exist somewhere in between these two extremes, with interoperability being a key challenge. Solutions such as Apache Arrow and Lakehouse aim to address this issue by allowing efficient exchange of relational data between systems. The popularity of each approach will depend on the specific needs and problems people are trying to solve.

Company
Fivetran

Date published
May 26, 2022

Author(s)
George Fraser

Word count
1060

Hacker News points
1

Language
English


By Matt Makai. 2021-2024.