Run AI workloads on any cloud, store on Hugging Face: zero-egress storage with SkyPilot

SkyPilot collaborates with Hugging Face to offer a seamless solution for running AI workloads across multiple cloud environments without incurring egress fees. This integration allows teams to keep their models and datasets on Hugging Face's Hub, while SkyPilot manages compute tasks on any cloud, using a single hf:// URL to mount repositories into SkyPilot jobs. The solution eliminates cross-cloud data transfer costs, as Hugging Face charges no egress fees, and leverages Xet-backed deduplication to efficiently manage storage by only transferring changed data chunks. This joint effort also enables SkyPilot to function as a backend for Hugging Face Storage, facilitating tasks like reading and writing to cloud object stores, with the added benefit of on-disk caching to optimize data access. This setup helps teams avoid the need to duplicate data across different cloud vendors, thus reducing idle GPU capacity and enhancing the operational efficiency of AI projects.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.