August 2023 Summaries

Large Language Models (LLMs) have captured significant interest, particularly following the launch of OpenAI's ChatGPT in 2022, leading enterprises to explore their potential for competitive advantage. A survey of over 150 professionals across 29 countries revealed that while many organizations are experimenting with LLMs, they face challenges like sharing proprietary data and high costs associated with commercial models. As a result, a growing number of companies prefer using open-source LLMs, which offer more control over data and are cost-effective. Key hurdles in deploying LLMs include data privacy, customization, training costs, hallucinations, and latency issues. The report suggests solutions such as hosting open-source LLMs in private clouds, using configuration-driven tools, and employing techniques like Retrieval-Augmented Generation (RAG) to enhance model reliability. There is a strong push towards building customized LLMs on open-source platforms to maximize the value and operational efficiency of AI investments, with platforms like Predibase emerging to support these needs.

Large Language Models (LLMs) can be effectively used for tasks traditionally handled by gradient-boosting models, such as predictions on tabular data, through the platform Predibase. Predibase allows users to connect various data sources and train models using neural networks or gradient-boosted trees. The process involves setting up a model repository, selecting input features, configuring the model, and using templates to serialize tabular data into a format suitable for LLM processing. Subsequently, users can prompt the LLM to convert these features into natural language representations, facilitating LLM serialization. The results, which can be exported as CSV files, are then used to train a neural network on the serialized data. Predibase's platform simplifies machine learning by providing visualizations of model performance metrics and utilizing a Declarative ML approach, which reduces the complexity typically associated with ML processes. The article emphasizes that Predibase's capabilities extend beyond the showcased application, encouraging users to explore the platform further through resources such as blog posts, webinars, and free trials.

Large Language Models (LLMs) are being explored for their potential to handle tabular data tasks traditionally dominated by models like Gradient Boosting Machines (GBMs). The "TabLLM" paper investigates the feasibility of using LLMs for tabular classification by converting data into text prompts, allowing LLMs to process it as natural language. The study found that while LLMs can perform well, especially in low-data scenarios, they face challenges such as limited context length and reliance on meaningful column semantics. The experiments revealed that LLMs could match or exceed the performance of GBMs in some fully fine-tuned settings, particularly on datasets with fewer features, though GBMs remain preferred for larger, data-rich tasks due to their efficiency and cost-effectiveness. The analysis underscores the strengths and limitations of LLMs, suggesting they are a viable option for tabular tasks when data is scarce, but their suitability depends on factors like data richness and feature nature.

Ludwig v0.8, an open-source, low-code framework originally released by Uber, introduces a suite of features designed to facilitate the customization and fine-tuning of large language models (LLMs) for both generative and predictive tasks. This version allows developers to leverage tools such as the "llm" model type for creating text-based AI systems like chatbots and code assistants, with compatibility for HuggingFace transformers. It also integrates with Deepspeed to handle large models across multiple GPUs, incorporates parameter efficient fine-tuning techniques like Low-rank adaptation (LoRA), and supports quantized training on a single GPU. Ludwig v0.8 enhances prompt templating for context-specific model responses, zero-shot, and in-context learning to minimize labeled data requirements, and introduces retrieval-augmented in-context learning for improved prediction performance. The release aims to streamline the process of building and deploying LLMs by resolving infrastructure challenges, allowing users to concentrate on model development. Additionally, Ludwig v0.8 offers new integrations with technologies such as Daft for improved preprocessing speed and compatibility with PyTorch 2.0, encouraging community engagement and contributions for future enhancements.