November 2024 Summaries

Replicate has introduced optimizations to significantly accelerate fine-tuning FLUX models with user data, achieving speeds comparable to base models, with the improvements being open-source. The adjustments involve utilizing Alex Redden’s flux-fp8-api, torch.compile, and CuDNN attention kernels, and support loading LoRAs from sources like Hugging Face and Civitai. Fine-tunes are quantized as fp8 and merged with the base model, with an automatic increase in lora_scale for optimal output when go_fast=true is enabled. While quantization slightly alters outputs, it minimally affects quality, and all models, including existing and future ones, will benefit from these enhancements. Acknowledging the challenge of comparing model outputs, Replicate emphasizes transparency and the importance of contributing optimizations back to the open-source community, with a commitment to making both fine-tuning and training processes faster through ongoing developments and community collaboration.

Black Forest Labs has introduced FLUX.1 Tools, an innovative suite enhancing control and steerability for their FLUX text-to-image model, now available on Replicate. This collection includes four new features: Fill, which allows for inpainting and outpainting to edit images precisely; Canny, which uses edge detection for creating images with clear structure; Depth, which employs depth maps to render images with realistic perspectives; and Redux, an adapter enabling variations of images with customizable text prompts. These tools are designed to work across FLUX.1 [dev], [pro], and [schnell] models, with Redux also set to expand to FLUX.1 [pro] 1.1 and pro 1.1 ultra models. Additionally, ControlNet, an open-source tool, is leveraged to guide image generation using conditioning images and text prompts, facilitating the creation of structured and creatively detailed outputs.

NVIDIA L40S GPUs have been introduced as a new hardware option, replacing the older A40 GPUs with a performance boost of around 40%, while maintaining the same cost for private models and deployments despite their greater expense per second. The transition to L40S GPUs will occur over several weeks, with public models migrating by early December and private models shortly after. Users can immediately switch their models and deployments to L40S GPUs, with the option to continue using A40 GPUs temporarily until the migration is complete. While the L40S GPUs are pricier, the increased speed can result in cost savings or at least maintain current expenses due to reduced processing time. Users are advised to adjust their deployments accordingly to optimize performance and cost, and further details on benchmarking and pricing are available in the company's resources.