Choosing an inference stack
Choose the runtime after the hardware and workload are known. MLX is the first check on Apple Silicon, llama.cpp is the portable default across GPU and CPU hardware, ExLlamaV3 targets single-user NVIDIA inference, and vLLM or SGLang are serving choices. A large MoE model that needs CPU expert offload calls for a specialist hybrid path.
Formula
stack = f(platform, full-memory fit, single-user or serving, dense or MoE)
Worked example: Qwen3-Coder-30B-A3B on M4 Pro
The artifact's Apple anchor measures about 130 tps with the MLX backend, compared with 43 tps on the older llama.cpp backend. On this hardware and model, the runtime choice changes the answer more than a small tuning knob.
MLX: ≈130 tps older llama.cpp backend: 43 tps