MoE vs dense models
A dense model reads every parameter for each token. A mixture-of-experts model stores all experts in memory but routes each token through only a subset. That means total parameters decide whether the weights fit, while active parameters decide most of the decode work. Mixing up those two numbers can understate MoE speed by 5 to 10 times.
Formula
MoE fit uses total params; decode bytes ≈ active params × bpw ÷ 8 + KV reads
Worked example: Qwen3-30B-A3B on an RTX 3090
Qwen3-30B-A3B stores 30.5B parameters but reads about 3.3B active parameters per token. The measured anchor is 101 tps on one RTX 3090, so it decodes like a much smaller model while still needing room for all 30.5B parameters.
fit: 30.5B total params decode: 3.3B active params → 101 tps