MTP and speculative decoding

Multi-token prediction, or MTP, uses draft heads shipped inside the model to propose several tokens per step; classic speculative decoding uses a separate draft model for the same trick. Both attack the same bottleneck, so llm.fit takes the better one rather than multiplying them. The payoff is an acceptance rate, not a constant, because it depends on how predictable the text is: 1.7 times on Qwen3.6-27B, 1.8 on DeepSeek-V3. llm.fit plans at 1.8 and widens the band to cover the spread.

Formula

tps_{accelerated} = tps_{base} \times α_{accepted}

Worked example: Qwen3.6-27B with MTP

38 tokens per second becomes 65 on an RTX 3090. That 1.71 multiplier is an acceptance rate, not a hardware property: it measures how often the drafted tokens survive verification, so predictable text does better than surprising text on identical silicon.

\frac{65}{38} = 1.71 \times

Try MTP on a current Qwen 27B