Hugging Face: the transformers backend in vLLM now runs at native speed

On July 8, 2026, Hugging Face published benchmark results on its blog: the transformers library backend in vLLM now fully matches the performance of native vLLM implementations for compatible architectures. Developers only need to add a single `--model-impl transformers` flag when launching the server, with no changes to their infrastructure.

What changes for developers

Until now, high-performance inference in vLLM required separate hand-written implementations for each architecture. The research version of a model in transformers and its production version in vLLM diverged, and every architecture update meant synchronizing two codebases.

Now the transformers library is automatically compiled into optimized vLLM kernels via `torch.fx` (static analysis of the computation graph) and AST manipulation to rewrite operations. Under the hood, the system builds a computation graph, looks for patterns that can be replaced with a single optimized kernel and, for models using tensor parallelism, additionally fuses column-parallel and QKV-parallel layers.

Key facts:

• Date: July 8, 2026, official Hugging Face blog
• Launch flag: `--model-impl transformers` in the `vllm serve` command
• Test models: Qwen3-4B (1 GPU), Qwen3-32B (2 GPUs, tensor parallelism), Qwen3-235B MoE FP8 (8×H100)
• Result: the backend matches or exceeds native vLLM throughput
• Compatibility: `torch.compile`, CUDA Graphs, training support (absent from native vLLM implementations)
• Coverage: 450+ architectures in the transformers library

How it was tested

Hugging Face benchmarked three Qwen3 configurations of increasing complexity: Qwen3-4B on one GPU, Qwen3-32B with tensor parallelism on two GPUs, and Qwen3-235B in FP8 with a Mixture-of-Experts architecture on eight H100s, using data parallelism and expert parallelism at the same time. In all three configurations, the transformers backend delivered throughput on par with or above native vLLM.

"Model developers can now get ultra-fast vLLM inference for free," — from the official

Hugging Face blog.

There is one important advantage over native vLLM implementations: the transformers backend keeps training support. Native vLLM implementations were designed exclusively for inference, so teams that needed both fine-tuning and high-performance serving previously had to maintain two separate code paths.

Why it matters for the ecosystem

The transformers library serves as the reference implementation for 450+ architectures and is integrated into SGLang, MLX and llama.cpp. Parity with native vLLM removes one of the main arguments against using it in production: the need to rewrite models for a specific serving engine. This matters especially given fragmentation, since each engine previously required its own implementation.

The path from publishing a new model on the Hugging Face Hub to a high-performance deployment gets considerably shorter. Developers no longer have to wait for the vLLM team to add native support for an architecture; a flag and a compatible implementation in transformers are enough. This is especially important for independent researchers and small teams that lack the resources to write and maintain separate vLLM implementations.

Current limitations: models with linear attention are not yet supported; custom models from Hub repositories may run into compatibility problems.

What this means

The barrier between research code in transformers and high-performance inference in vLLM has practically disappeared. What used to require manual optimization for each engine now takes a single command-line flag — a direct cut in engineering costs for teams building and deploying large language models.

Frequently asked questions

Which models does the new backend support?

The backend supports 450+ architectures from the transformers library. It was tested on Qwen3-4B, Qwen3-32B and Qwen3-235B MoE FP8. Models with linear attention are not supported in the current version; custom models from the Hub may need a compatibility check.

How do I run vLLM with the transformers backend?

For a single GPU: `vllm serve Qwen/Qwen3-4B --model-impl transformers`. For multiple GPUs: `vllm serve Qwen/Qwen3-32B --model-impl transformers --tensor-parallel-size 2`. No other infrastructure changes are required.