Конфиденциальный инференс на NVIDIA H100 под Intel TDX: приватность стоит 17–21% скорости

Researchers published a benchmark on arXiv in July 2026 that measured the cost of confidential computing for LLM inference: on a single NVIDIA H100 80GB GPU inside an isolated Intel TDX environment, confidential mode reduces global throughput by 17.7–21.1% and slows time to first token by 21.8–27.8%.

What exactly was measured

The authors compared two execution modes on a single NVIDIA H100 80GB GPU hosted in a confidential Intel TDX instance: normal (non-confidential) and protected (confidential computing). For the test they took two representative language models — Mistral-7B v0.1 and Qwen3-30B-A3B — and measured five metrics: time to first token (TTFT), end-to-end request latency, token generation speed, global throughput, and the number of served requests as concurrency increases.

• Hardware — one NVIDIA H100 80GB in a confidential Intel TDX instance
• Models — Mistral-7B v0.1 and Qwen3-30B-A3B
• TTFT increases by 21.8% (Mistral-7B) and 27.8% (Qwen3-30B-A3B)
• Global throughput drops by 17.7% and 21.1%
• Under concurrent load the gap is 11.5–20.2%

How much does performance drop?

In experiments with fixed request intensity, confidential mode increased the average time to first token by 21.8% for Mistral-7B and by 27.8% for the larger Qwen3-30B-A3B. Global throughput in tokens dropped by 17.7% and 21.1% respectively.

The gap is more noticeable for the larger model: the bigger the deployed network, the more expensive computation isolation becomes. According to the study, the overhead comes from memory encryption and protected data exchange between the CPU and GPU inside the trusted environment.

Why this matters for capacity planning

Under growing concurrent (closed-loop) load, the throughput gap stays in the range of 11.5–20.2%, but the larger Qwen3-30B-A3B hits its saturation point earlier in confidential mode. That means the same server in confidential mode will serve fewer simultaneous users before latency starts rising sharply.

"Confidential GPU inference preserves usable throughput under load,

but capacity planning must account for both a persistent performance penalty and earlier saturation of larger models," the arXiv paper states.

What it means

Confidential computing is no longer just theory — it's becoming a practical requirement for inference on sensitive data or proprietary models. The privacy tax — roughly a fifth of throughput — is measurable and, according to the authors' findings, acceptable if factored into capacity planning in advance.

Frequently Asked Questions

What is confidential GPU inference?

It's a mode in which input data and model weights remain encrypted and inaccessible even to the server operator. It's needed when inference runs on sensitive data or protects proprietary models from leaking — exactly the scenario the authors tested on NVIDIA H100 under Intel TDX.

How much LLM performance is lost in confidential mode?

According to the arXiv measurements, global throughput drops by 17.7–21.1%, while time to first token increases by 21.8–27.8%. Under concurrent load the gap stays within 11.5–20.2%, with larger models hitting the ceiling sooner.