NVIDIA L40S GPUs are here

NVIDIA L40S GPUs have been introduced as a new hardware option, replacing the older A40 GPUs with a performance boost of around 40%, while maintaining the same cost for private models and deployments despite their greater expense per second. The transition to L40S GPUs will occur over several weeks, with public models migrating by early December and private models shortly after. Users can immediately switch their models and deployments to L40S GPUs, with the option to continue using A40 GPUs temporarily until the migration is complete. While the L40S GPUs are pricier, the increased speed can result in cost savings or at least maintain current expenses due to reduced processing time. Users are advised to adjust their deployments accordingly to optimize performance and cost, and further details on benchmarking and pricing are available in the company's resources.