Hugging Face: transformers backend in vLLM now runs at native speed

On July 8, 2026, Hugging Face published benchmark results in their blog: the transformers library backend in vLLM now fully matches native vLLM implementations in performance for compatible architectures — developers only need to add a single flag `--model-impl transformers` when launching the server, with no infrastructure changes.

What changed for developers

Previously, high-performance inference in vLLM required separate "manual" implementations for each architecture. The research version of a model in transformers and its production version in vLLM diverged and required syncing two codebases with each architecture update.

Now the transformers library automatically compiles to optimized vLLM kernels via `torch.fx` (static graph analysis) and AST manipulations for operation rewriting. Under the hood, the system builds a computation graph, searches for patterns to replace with a single optimized kernel, and for models with tensor parallelism additionally fuses column-parallel and QKV-parallel layers.

Key facts:

• Date: July 8, 2026, official Hugging Face blog
• Launch flag: `--model-impl transformers` in `vllm serve` command
• Test models: Qwen3-4B (1 GPU), Qwen3-32B (2 GPU, tensor parallelism), Qwen3-235B MoE FP8 (8×H100)
• Result: backend equals or surpasses native vLLM in throughput
• Compatibility: `torch.compile`, CUDA Graphs, training support (absent in native vLLM implementations)
• Coverage: 450+ architectures in transformers library

How testing was done in practice

Hugging Face benchmarked three Qwen3 configurations, escalating in complexity: Qwen3-4B on a single GPU, Qwen3-32B with tensor parallelism on two GPUs, and Qwen3-235B in FP8 format with Mixture-of-Experts architecture on eight H100s with simultaneous data parallelism and expert parallelism. On all three configurations, the transformers backend showed throughput at or above native vLLM.

"Developers can now get ultra-fast vLLM inference for free," — from official

Hugging Face blog.

An important advantage over native vLLM implementations: the transformers backend preserves training support. Native vLLM implementations were designed exclusively for inference — teams needing both fine-tuning and high-performance serving previously had to maintain two separate code paths.

Why this matters for the ecosystem

The transformers library serves as a reference implementation for 450+ architectures and is integrated into SGLang, MLX and llama.cpp. Parity with native vLLM eliminates one of the key arguments against its production use — the need to rewrite models for a specific serving engine. This is especially significant in the context of fragmentation: each engine previously required its own implementation.

The path from publishing a new model on Hugging Face Hub to high-performance deployment substantially shortens. Developers no longer need to wait for the vLLM team to add native architecture support — a flag and compatible transformers implementation are enough. This is especially important for independent researchers and small teams without resources to write and maintain separate vLLM implementations.

Current limitations: models with linear attention are not yet supported; custom models from Hub repositories may encounter compatibility issues.

What it means

The barrier between research code in transformers and production inference in vLLM has virtually disappeared. What once required manual optimization for each engine is now achieved with a single command-line flag — direct reduction in engineering costs for teams developing and deploying large language models.

Frequently Asked Questions

Which models does the new backend support?

The backend supports 450+ architectures from the transformers library. Tested on Qwen3-4B, Qwen3-32B and Qwen3-235B MoE FP8. Models with linear attention are not supported in the current version; custom models from Hub may require compatibility verification.

How do I run vLLM with the transformers backend?

For one GPU: `vllm serve Qwen/Qwen3-4B --model-impl transformers`. For multiple GPUs: `vllm serve Qwen/Qwen3-32B --model-impl transformers --tensor-parallel-size 2`. No other infrastructure changes required.