August 2025 Summaries

Deploying AI models into self-hosted or air-gapped environments presents distinct challenges, particularly when dealing with large language models (LLMs). H2O.ai has developed an architecture that separates model artifacts from serving infrastructure to facilitate deployment without compromising performance. Key components include a self-contained Kubernetes environment, a service for managing application components, and an object storage service for large model artifacts. The architecture supports various deployment options, allowing customers to integrate their own storage solutions or use cloud-based services. H2O.ai emphasizes best practices such as differentiating model sizes for packaging, designing for offline environments, using Helm or Replicated for installation, and storing large models externally to avoid overloading container registries. They also focus on fine-tuning and model ownership, optimizing model-serving parameters, and ensuring dev/prod parity through CI/CD pipelines. This approach offers a flexible and reliable way to distribute AI models in demanding enterprise environments.

Replicated's August 2025 release introduces several updates across its platforms, including a preflight check in Embedded Cluster 2.7.1 to prevent installations on incompatible XFS filesystems, and a Beta feature for the Vendor Portal allowing vendors to offer self-service licenses through an Enterprise Portal sign-up URL. The Vendor Portal now includes a version flag in Helm installation instructions, and an invite user modal is automatically displayed when enabling the Enterprise Portal for new customers. Documentation updates include a guide on setting up the environment with essential tools like kubectl and helm, an Enterprise Portal diagram, instructions for configuring the HelmChart v2 custom resource builder key for air gap installations, and descriptions of Unix accounts created during Embedded Cluster installs. These enhancements aim to improve user experience, facilitate software deployment, and provide comprehensive resources for working with the Replicated platform.