July 2024 Summaries

The rapid growth in Small Language Models (SLMs) has led to an increase in confusion around model licensing. While open-weight models allow access to the weights of neural networks, they often come with restrictions on modification, distribution, and commercial use. This is different from truly open-source models that permit free usage, modification, and distribution for any purpose. The Hugging Face platform hosts 784,000 open-weight models, but only 25% of them are open-source due to license confusion. Companies must pay attention to licensing when generating synthetic data with these models to ensure data ownership is respected.

Gretel Navigator is a compound AI system that enables users to easily create high-quality synthetic data for training AI and LLMs. It leverages agentic workflows, task planning, and multiple tools & models to iteratively review and improve synthetic data. Key features include evolutionary algorithms, multi-LM collaboration, agent-based generation, self-alignment, comprehensive governance, and customizable design. Gretel Navigator can generate a wide variety of synthetic data, including text, instruction-response pairs, step-by-step evaluation data, and question-answer pairs from documents. It has been shown to outperform its own underlying LLMs and even much larger models such as OpenAI's GPT-4.

Gretel has released a comprehensive "Synthetic Multilingual LLM Prompts" dataset, featuring 1,250 synthetic prompts in seven languages. The dataset is designed for use with conversational LLMs like ChatGPT and is available on GitHub and Hugging Face. Translation quality was assessed using the LLM-as-a-Judge method, ensuring accuracy, fluency, and consistency across languages. This dataset is released under the Apache 2.0 license and can be used with proper attribution.