Inside Cohere Model Vault: Building a verifiable confidential AI inference

Cohere has introduced Confidential Computing on Model Vault in public beta, using NVIDIA H100 GPUs on Microsoft Azure to protect sensitive AI inference workloads through hardware-backed isolation, encrypted CPU-to-GPU communication, encrypted application payloads, and remote attestation. The system uses Oblivious HTTP to ensure prompts and responses remain encrypted through intermediary infrastructure until they reach an attested confidential environment, while customer-side policy verification determines whether the environment’s measured hardware, firmware, software, and GPU security state are acceptable before data is released. This approach aims to reduce the infrastructure that must be trusted with plaintext data, protecting workload memory from cloud operators, hosts, hypervisors, and intermediaries, though it does not eliminate risks such as side-channel attacks, denial-of-service, application vulnerabilities, or observable metadata like timing and traffic volume. Cohere emphasizes that confidential computing complements rather than replaces application security, compliance, data-residency controls, and contractual safeguards, and plans to open-source its confidential inference stack to support independent inspection and reproducible verification. The company is also developing support for NVIDIA Blackwell B200 infrastructure as it moves toward general availability.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.