ExecuTorch brings PyTorch to browser GPUs: text generation is 2–10 times faster

If your team trains models in PyTorch, running them in the browser usually adds another engineering task: converting them to a suitable format and checking that everything still works afterward. ExecuTorch's new WebGPU backend shortens that path. The model is exported to a .pte file, and computation runs on the user's GPU. In the published test, text generation was 2–10 times faster than the CPU version. For developers, this is a reason to investigate how much processing they can move from the server into a browser tab.

What changes for the team

Imagine a web application where users upload a product photo and remove its background. Server-side processing means receiving the image, queuing the task, running the model, and returning the result. Local processing allows a different workflow: the browser downloads the model, then processes the photo on the user's device. This is an example of how the technology could be used, not a benchmark of an existing commercial service.

The implications for the team are concrete. If the client device can handle the processing, there is no need to pay for a server-side model run for every photo. However, other costs arise: delivering model weights, supporting different devices, and troubleshooting browser errors. Savings need to be calculated using your own traffic, rather than assumed simply because WebGPU is available.

Under the hood, the backend combines WebAssembly and WebGPU. The former enables the runtime to operate in the browser; the latter provides access to GPU computation. Export stays within the PyTorch toolchain: there is no need to switch to another framework's format. This is still a separate model preparation step, not a way to drop arbitrary Python code into a web page. The architecture and test results are described in the PyTorch team's post.

Where the 2–10 times speedup comes from

The team compared WebGPU with ExecuTorch's own CPU backend, which uses WebAssembly and XNNPACK. Measurements were taken in Chrome on an Apple M1 Pro using quantized Llama 3.2 1B and Qwen3-0.6B models. Input context processing was 4–7 times faster, while sequential token generation was 2–10 times faster. These figures apply to this specific comparison, not to every way of running models in a browser.

For product teams, it is important to distinguish two sources of latency. First, the application must download the model and prepare it to run. Then the user waits for a response to their request. Faster computation helps, but on its own it does not tell you how long someone will stare at the screen when opening the application for the first time. That is a separate metric to measure in a prototype.

A practical test should therefore focus on a user action: open the page over a typical network connection, upload a real file, get a result, and repeat the operation. Compare the first run with subsequent runs, check available memory, and test behavior on a low-powered laptop. A fast response on a developer's machine does not yet show what the experience will be like for the entire audience.

What to try and where the limits are

You can start with the official Playground. It offers local chat, speech recognition, object segmentation, and depth estimation from camera images. The playground is labeled experimental; it is a convenient way to test the workflow on your own devices before preparing your own export.

For applications involving photos or voice, local processing also changes the data flow: inputs do not need to be sent to a server for inference. But this depends on the specific implementation. If you add uploads of source files to analytics or a cloud fallback, you will need to revisit the promise you make to users. The processing architecture and actual network requests need to be checked together.

The backend itself is still at an early stage of development. A sensible next step is therefore to pick one narrowly scoped feature, define the supported devices, and compare it with your current solution. For example, test background removal on typical customer photos. The deployment decision should be based on time to result, processing quality, and support costs. The new path from PyTorch to the browser makes that experiment easier, but the team must measure the outcome itself.