Fine-Tuning CodeLlama on Gretel & AWS SageMaker JumpStart

In this blog post, the authors discuss fine-tuning CodeLlama models on Gretel's synthetic Text-to-SQL dataset using AWS SageMaker JumpStart. They demonstrate how to prepare the dataset and create an instruction prompt template for fine-tuning. The fine-tuned model is then evaluated on the BIRD benchmark, showing significant improvements in execution accuracy (EX) and valid efficiency score (VES). This highlights the potential of synthetic datasets in enhancing LLMs for specialized tasks like Text-to-SQL parsing. A SageMaker notebook for this blog post is available, along with Gretel's platform on AWS Marketplace.