May 2023 Summaries

Cloud platforms like AWS, GCP, and Azure provide robust managed services essential for machine learning (ML) workloads but can lead to high costs if not properly managed. Effective cost management begins with tracking and measuring expenses, enabling identification of costly models, teams, or datasets. Key strategies to control costs include implementing compute and data caching, utilizing checkpoints to recover from failures without restarting from scratch, and colocating data and compute resources to minimize expensive data transfers. GPU utilization can be maximized by optimizing data loading and memory management, employing GPU-optimized libraries, and using asynchronous operations. Infrastructure-level optimizations involve selecting the right cloud provider and pricing model, using spot instances or preemptible VMs, and leveraging auto-scaling for dynamic resource allocation. Beyond cloud costs, optimizing human resource expenses through strategic project prioritization, tool selection, and knowledge sharing can significantly impact overall ML cost efficiency.

Sematic's latest release, version 0.29.0, introduces new features, reliability improvements, and bug fixes, focusing on enhancing user control and experience. Key updates include pipeline metrics for assessing run count, success rate, and average runtime, as well as the introduction of Function timeouts to manage resource-intensive processes effectively. The update also simplifies terminology by renaming Non-Inline Functions to Standalone Functions, which operate in dedicated Kubernetes Pods, and allows customization of local storage locations to better support on-premise deployments. Other enhancements include a new Pod lifecycle tab for easier debugging, backward log scrolling, Ray autoscaling for cluster management, and a garbage collection cron job to maintain data integrity. Users can upgrade to the new version seamlessly and are encouraged to engage with the Sematic community for support.