December 2023 Summaries

Gretel's Tabular LLM is an innovative solution designed to address sparse data issues commonly encountered in tabular datasets. The model leverages the generative capabilities of large language models (LLMs) to create contextually relevant and high-quality responses, which can be used to fill in missing values in a table while maintaining its structured format and preserving field and row-level correlations for numerical, categorical, and free text data types. The approach is particularly useful for e-commerce product listings where multiple constraints need to be reflected in the synthetic insertions. Prompt engineering techniques can further enhance the results by iterating on producing better outputs. Gretel's Tabular LLM early preview is now available for users interested in testing this example or experimenting with their own applications.

Gretel Tuner is a new config-driven tool for efficiently tuning hyperparameters of Gretel Synthetics models. It's integrated into the Python SDK, offering simplicity through YAML configuration and leveraging Optuna’s Bayesian optimization framework for efficiency. The tool also supports custom user-implemented optimization metrics, allowing users to tune it according to their specific use cases. Google Colab tutorials are available for getting started with Gretel Tuner, which can be installed locally by adding the [tuner] option during SDK installation.

Gretel has announced a partnership with Microsoft Azure and joined the Microsoft for Startups Pegasus Program. This collaboration aims to promote responsible AI practices across industries by offering synthetic data generation platforms and privacy-preserving technologies on Azure Marketplace. The integration of synthetic data into AI and data governance frameworks is reshaping strategies, enabling organizations to mitigate privacy risks, enrich limited data reservoirs, address dataset biases, uphold compliance with global data protection regulations, and support best practices in data governance, ethics, and fairness.

The text discusses how to generate diverse, high-quality synthetic data for training better Language Learning Models (LLMs) and Small Language Models (SLMs). It mentions that recent research has shown that SLMs trained on such data can achieve state-of-the-art results. Techniques like including random word subsets in prompts are used to create diverse datasets. The text also highlights the advantages of using textbook-like data for training models, as it leads to efficient knowledge storage and reduced toxic content generation. To get started with this approach, users need a Gretel API key, access to Gretel's Tabular LLM, and domain-specific training data. A Colab notebook and video walkthrough are provided for guidance.