Qwen3.6-35B-A3B hits 109 tok/s on an RTX 5080 with an RTX 3060 on PCIe Gen2 x4

A Habr author ran Qwen3.6-35B-A3B on an RTX 5080 paired with an RTX 3060 and got 109 tokens per second during generation. The second card sits in a PCIe Gen2 x4 slot. That is about 2 GB/s in theory, less than any budget NVMe drive delivers. The link had no effect on generation speed.

The original Habr post is available in RSS only as a teaser. It does not disclose the runtime configuration or the quantization, so below we explain the mechanics behind the result and flag our own estimates separately.

Why a narrow slot doesn't choke generation

Gen2 bandwidth is easy to work out. One lane runs at 5 GT/s, which leaves roughly 500 MB/s after 8b/10b encoding. Four lanes give about 2 GB/s. On paper that is little: pushing a model's full weights through such a link would take painfully long.

But when a model is split by layers across cards, the weights don't travel. They are loaded into VRAM once and stay there. At each step, only the hidden-state vector of the current token passes between GPUs. Its size is measured in kilobytes, so the per-token traffic is tiny. For this kind of load, latency matters more than link width, and Gen2 latency is negligible next to the compute time of the layers.

What the MoE architecture contributes

The A3B suffix in the name means that out of roughly 35B parameters, about 3B are active for each token. The remaining experts simply sit in memory. For decoding, that means fewer weights to read per token, and generation speed usually runs into memory bandwidth first.

A rough estimate: with 4-bit quantization, about 1.5–2 GB is read per token. At 109 tok/s, that works out to around 160–220 GB/s of effective read bandwidth. The RTX 5080 with 16 GB of GDDR7 has a peak bandwidth of about 960 GB/s, and the RTX 3060 with 12 GB of GDDR6 has about 360 GB/s. The numbers fit without any miracles. This is our estimate, not data from the author.

Together, the pair has 28 GB of VRAM. That is enough for a 35B model in 4-bit form: the weights take about 18–20 GB, leaving headroom for the KV-cache.

Key parameters

• Model: Qwen3.6-35B-A3B, MoE, about 35B parameters, about 3B active per token
• Primary card: RTX 5080, 16 GB GDDR7
• Second card: RTX 3060, 12 GB GDDR6
• Second card's link: PCIe Gen2 x4, about 2 GB/s in theory
• Generation speed: 109 tok/s

How to use this in practice

The main takeaway for owners of older hardware: a second GPU in a "bad" slot is not useless. If you have an old 3060 or a similar card lying around, you can connect it through a chipset x4 slot and gain extra gigabytes of VRAM for a large MoE model.

A few caveats follow. They cover cases the teaser does not describe.

• The split mode matters. A layer split puts almost no load on the bus. Tensor parallelism shuttles data between the cards at every layer, and on Gen2 x4 it will slow down noticeably.
• Prompt processing (prefill) stresses the bus more than generation does. If you feed the model long contexts, measure time to first token separately from tok/s.
• Distribute layers unevenly. Give the fast card more layers, and give the slow one only as many as don't fit into 16 GB.
• Check the actual link width in GPU-Z or with lspci. Some slots labeled x16 actually run as x4 or even x1.

The bottom line is simple: for decoding sparse models, VRAM capacity and speed matter more than the width of the bus between cards. Before buying a second card for a top-end platform, check whether the slot you already have is enough.