NVIDIA releases Nemotron 3.5 Bolt E4B: 4 GB and 133 tokens per second on an RTX 5070

NVIDIA has published the weights of Nemotron 3.5 Bolt E4B, the most compact model in the Nemotron 3.5 lineup, designed to run locally on GeForce RTX GPUs, Jetson boards and Spark mini-computers. At 4-bit quantization it takes about 4 GB of memory and generates 133 tokens per second on an RTX 5070, with the first token arriving after 548 ms. The release came a little over a week after the announcement of the multimodal Nemotron 3.5 Super VL and its compressed 74-billion-parameter Bolt version. NVIDIA promised a small model at the time, and it is now openly available.

Not a compressed bigger model, but a fine-tuned Gemma

E4B has a different origin than the larger Bolt. The 74-billion-parameter version was produced by compressing Super VL. The small model is built on Google's Gemma 4 E4B: NVIDIA fine-tuned it with reinforcement learning for instruction following, multi-step tool calling and dialogue.

The model has 8 billion parameters in total, but 4 billion are active for each token. The context window is 64 thousand tokens, and 35 languages are supported. Reasoning can be switched on and off, and its length can be capped for speed. For anyone running a local model, that is an important lever: a chain of reasoning consumes tokens and time, and on weak hardware it often makes sense to trim it.

What the benchmarks show and where the model loses

According to NVIDIA, Bolt E4B is on average 40% better than the previous Nemotron 3 Nano 4B and uses up to 80% fewer tokens per response. It outperforms its parent, Gemma 4 E4B, almost across the board. All of these figures are the company's own measurements, and there is no independent verification yet.

Qwen3.5 4B remains stronger. On Tau2-Bench (a benchmark of agentic dialogues with function calling), which tests tool use, it scores 61.7 against 52.3 for the new model. On RULER (a set of tasks on finding and using information in long text), which tests long context, the gap is just as noticeable: 95.4 against 85.8. If you need reliable tool calling or work with long documents on a small model, choosing Bolt E4B is not an obvious call.

Speed depends heavily on the hardware. On a Jetson Orin Nano, you wait 5.8 seconds for the first token, and generation then runs at 15.5 tokens per second. That is tens of times slower than the RTX 5070 on latency and nearly nine times slower on throughput. For an interactive assistant on the Orin Nano, that response time may be borderline.

DFlash draft models and runtime support

Alongside the BF16 and GGUF weights, NVIDIA released separate auxiliary draft models built with the DFlash method. They propose several next tokens at once, and the main model verifies them. NVIDIA says this yields up to a 21% speedup. With this approach, answer quality is determined by the main model, and the gain comes from parallel verification.

Support is already available in llama.cpp, Ollama, LM Studio, vLLM and SGLang, so the model can run on a laptop or in a server stack without custom modifications. The license is OpenMDW 1.1. The weights come with a blog post and a technical report.

The takeaway for practitioners is this. Bolt E4B looks like a good candidate for RTX and Spark applications that need fast response, multilingual support and controllable reasoning within a modest 4 GB of memory. For scenarios with heavy agentic chains or long context, first run the model on your own tasks and compare it with Qwen3.5 4B. And on the Jetson Orin Nano, check the real latency before shipping to production.