Cohere launches attested confidential inference on H100 GPUs in Azure

Cohere has opened a public beta of Confidential Computing for Model Vault, its dedicated managed inference service. Requests and responses are now protected not only at rest and in transit, but also during computation: the model runs on NVIDIA H100 GPUs in Microsoft Azure infrastructure inside a hardware-isolated environment. Clients can also verify that environment themselves before sending the first byte of data. The announcement was published on October 22, 2026, and you can sign up now at vault.cohere.com.

The third state of data

Conventional encryption covers two states: data at rest and data in transit. The third, data in use, has been a blind spot for years. For a model to process a prompt, the prompt has to sit in memory in plaintext. In theory, that makes it accessible to privileged host software, administrators, and anything running below the application layer.

Confidential Computing in Model Vault closes exactly this gap. Inference runs in a trusted environment that covers both the processor and the GPU accelerator. Hardware isolation protects the workload's memory from everything outside the boundary of that environment, including privileged infrastructure software and operators. On the NVIDIA side, GPU memory is protected, and traffic between the CPU and GPU is encrypted while the model runs.

Key launch details:

• status: public beta; encrypted vaults can be created and administered through Model Vault;
• hardware: NVIDIA H100 with Confidential Computing;
• cloud: Microsoft Azure provides the underlying confidential infrastructure;
• operating model: Cohere runs the inference platform as a managed service;
• integration: the same APIs and familiar request formats, with protection built into the client layer and the infrastructure;
• encryption: the application payload is encrypted end-to-end to the attested environment, and responses are protected on the way back.

Attestation instead of taking their word for it

The most interesting part of the release is remote attestation. Before sending sensitive data, client software obtains cryptographically signed evidence about the environment that will process it: the hardware, firmware, and software configuration. That evidence is checked against an approved security policy. If the environment does not match the policy, the client simply does not send the request.

Cohere CTO Phil Blunsom puts it this way: customers get cryptographic proof that the workload is running in a protected environment, and they can verify that the data is inaccessible to outsiders, including Cohere itself. The trust model shifts from "trust us" to "verify it yourself."

A second layer of protection works along the request path. Encryption is usually limited to the connection level, so intermediate nodes such as load balancers and proxies can, in theory, see the content. Here the request content is encrypted to the attested environment. It passes through intermediate infrastructure in encrypted form, and only the trusted environment can decrypt it. The managed platform keeps its operational advantages but does not receive the plaintext of customers' requests.

The release was built together with partners. Vicki Bhatia of Microsoft stresses that sensitive workloads should be protected by more than contractual promises. John Fanelli of NVIDIA talks about the hardware guarantees on which platforms like Model Vault build confidential inference for enterprise scenarios.

Where this helps and where the limits are

Cohere explicitly names the classes of tasks that, because of security requirements, contracts, or regulation, often could not move to managed AI platforms:

• fraud investigations in the financial sector;
• search over privileged documents and due diligence in legal services;
• sensitive government information;
• patents and research materials;
• engineering and architectural documentation (EDD, IDD);
• search and analysis over proprietary source code.

The practical takeaway for security teams and CIOs: the question of whether you can hand this data to a model at all can now be settled with technical means, not only legal ones. Attestation is worth building into the procurement process as a verifiable requirement, and the policy of permitted configurations should be agreed with security staff in advance. Otherwise the client will block requests on formal grounds. The beta suits a pilot on one narrow use case, such as search over internal code or contracts, with latency and cost measured.

Cohere itself spells out the limits of the technology. Confidential computing does not eliminate all risks and does not replace application security, compliance, contractual measures, or data residency requirements. It is a hardware-backed, verifiable boundary around data and computation for the duration of inference, but not a universal shield. The company has put the details of the architecture, the path of an encrypted request, trust boundaries, and the threat model into a separate technical write-up, Inside Cohere Model Vault.