May 2022 Summaries

Predibase is an enterprise-grade platform offering a declarative approach to machine learning, designed to simplify and accelerate the process of operationalizing ML solutions for businesses. Created by a team with a rich background from companies like Uber, Google, and Apple, Predibase allows users to focus on specifying model outcomes rather than the implementation details, thereby reducing complexity and increasing accessibility. The platform supports seamless integration of structured and unstructured data, facilitates scalable model training, and enables easy deployment and continual improvement of models without additional engineering effort. Built on top of open-source tools like Ludwig and Overton, Predibase offers a unified configuration system that enhances collaboration among data scientists, engineers, and product teams. It introduces Predictive Query Language (PQL) to leverage SQL skills in ML tasks and is backed by a cloud-native, serverless infrastructure for efficient resource management. Currently available by invitation, Predibase aims to bring its innovative ML solutions to a wide range of industries, promising further developments and open-source contributions in the near future.

Ludwig v0.5.0 has expanded its capabilities from tabular datasets to text classification, leveraging AutoML to automatically create deep learning models given a dataset, label column, and time budget. This open-source framework uses pre-trained models like BERT-base to simplify the model search process while addressing the challenges of managing resources for text models. Through extensive experimentation and heuristic development, Ludwig AutoML is designed to produce models with an accuracy within 2% of those manually tuned by experts. The system is validated on diverse datasets and incorporates features like memory tuning and step-based evaluation to optimize performance under resource constraints. The framework aims to make deep learning accessible and iterative, with ongoing developments to extend its applications further, such as into image classification, while fostering community engagement and contributions.