As AI product demand increases, Cloudflare has developed Omni, a platform designed to efficiently manage AI models on its edge nodes by maximizing GPU usage. Omni allows multiple AI models to run on a single machine and GPU using lightweight isolation techniques, effectively improving model availability, minimizing latency, and reducing idle GPU power consumption. It achieves this by employing a single control plane to manage model instances, implementing process and Python isolation, and over-committing GPU memory to accommodate more models per GPU. This approach mitigates the challenges of managing infrastructure at scale, allowing for elastic scaling and fine-grained control over model lifecycles. Omni integrates into Cloudflare's internal routing and scheduling systems, offering a unified layer for diverse inference engines and supporting features like batching and function calling. By isolating models with distinct dependencies and optimizing memory management, Omni enhances the efficiency and performance of Cloudflare's Workers AI service, facilitating rapid deployment of new models and features.