Confidential Inference on NVIDIA H100 with Intel TDX: Privacy Costs 17–21% of Speed

Researchers published a benchmark on arXiv in July 2026 that measures the cost of confidential computing for LLM inference. On a single NVIDIA H100 80GB inside an Intel TDX isolated environment, secure mode reduces global throughput by 17.7–21.1% and slows time to first token by 21.8–27.8%.

What was measured

The authors compared two execution modes on a single NVIDIA H100 80GB GPU running in an Intel TDX confidential instance: regular (non-confidential) and secure (confidential computing). They tested two representative language models, Mistral-7B v0.1 and Qwen3-30B-A3B, and measured five metrics: time to first token (TTFT), end-to-end request latency, token generation speed, global throughput, and the number of requests served as concurrency grows.

• Hardware: one NVIDIA H100 80GB in an Intel TDX confidential instance
• Models: Mistral-7B v0.1 and Qwen3-30B-A3B
• TTFT rises by 21.8% (Mistral-7B) and 27.8% (Qwen3-30B-A3B)
• Global throughput drops by 17.7% and 21.1%
• Under concurrent load, the gap is 11.5–20.2%

How much does performance drop?

In experiments with a fixed request rate, confidential mode increased average time to first token by 21.8% for Mistral-7B and by 27.8% for the larger Qwen3-30B-A3B. Global token throughput fell by 17.7% and 21.1%, respectively.

The gap is wider for the larger model: the bigger the deployed network, the more expensive compute isolation becomes. According to the study, the overhead comes from memory encryption and secure data exchange between the CPU and GPU inside the trusted environment.

Why it matters for capacity planning

Under growing concurrent load (closed-loop), the throughput gap stays within 11.5–20.2%, but the larger Qwen3-30B-A3B hits its saturation point sooner in secure mode. In other words, the same server in confidential mode will serve fewer simultaneous users before latency starts to climb sharply.

"Confidential GPU inference preserves usable throughput under load,

but capacity planning must account for both a constant performance penalty and earlier saturation of larger models," the arXiv paper says.

What this means

Confidential computing is moving from theory to a practical requirement for inference on sensitive data or proprietary models. The price of privacy, about a fifth of throughput, is measurable and, in the authors' view, acceptable if it is built into capacity calculations up front.

Frequently asked questions

What is confidential inference on a GPU?

It is a mode in which input data and model weights stay encrypted and inaccessible even to the server operator. It is needed when inference runs on sensitive data or when proprietary models must be protected from leaks. This is exactly the scenario the authors tested on an NVIDIA H100 under Intel TDX.

How much performance does an LLM lose in secure mode?

According to the arXiv measurements, global throughput drops by 17.7–21.1%, while time to first token rises by 21.8–27.8%. Under concurrent load, the gap stays within 11.5–20.2%, and larger models hit their ceiling sooner.