Product Updates - March 2024

Recent advancements have significantly accelerated fine-tuning jobs, reducing processing time by 55-80% through optimizations such as shifting all SaaS customers to A100 GPUs and setting batch_size to Auto by default. An intuitive UI deployment page has been launched to manage serverless and dedicated deployments, offering features like event histories and detailed logs. Moreover, fine-tuned models can now be utilized if they pass a checkpoint, allowing teams to test mid-way and restart from checkpoints in case of failures, thus saving time and resources. Open Source LoRAX has been overhauled to prevent CUDA out of memory errors by automatically managing memory resources, alongside new enhancements in model architecture and batch size tuning for LLMs, which improve experimentation by enabling model weight loading from the latest training checkpoint.