Open models, such as DeepSeek-R1, are advancing in reasoning and coding capabilities, but deploying these models in production remains challenging due to their hardware requirements, like needing 16 H100 GPUs. The integration of Ray Serve LLM APIs with Kubernetes platforms, such as Google Kubernetes Engine (GKE) through KubeRay, aims to simplify this process by offering a developer-friendly, reliable, and cost-effective deployment solution. These APIs enhance large language model applications with features like autoscaling, load balancing, custom request routing, and robust observability, allowing for efficient multi-node model deployments. Ray Serve LLM's capabilities include scale-to-zero functionality, custom routing for optimized performance, and compatibility with OpenAI API, facilitating seamless integration with existing client implementations. The approach is designed to support complex models by distributing tasks across multiple nodes and GPUs, leveraging tensor and pipeline parallelism for better resource optimization and performance. The community-driven initiative encourages developers to engage with the AI infrastructure advancements without the burden of managing extensive infrastructure, promoting a collaborative environment for innovation.