MoE vs dense models

A dense model reads every parameter per token. A mixture-of-experts model keeps every expert resident but routes each token through a few, so total parameters decide whether it fits and active parameters decide how fast it decodes. Read the wrong one and you will understate MoE speed by 5 to 10 times: Qwen3-30B-A3B needs room for 30.5B and moves like 3.3B.

Formula

fit \leftarrow P_{total} decode bytes \approx \frac{P _{active} \times bpw}{8} + KV reads

Worked example: Qwen3-30B-A3B on an RTX 3090

This model stores 30.5B parameters and reads 3.3B per token. It needs a 30B model's memory and decodes at 101 tokens per second, like a model a ninth its size. Quote either number alone and you describe a different machine.

fit: 30.5 B decode: 3.3 B \frac{30.5}{3.3} = 9.2 \times fewer bytes per token

Try a current MoE model on an RTX 3090