Understand the numbers.

Short guides to the quantities that decide whether a model fits, how fast it runs, and which runtime makes sense.

Bits per weightHow quantization changes model size, memory use, and quality.Why bandwidth decides speedWhy single-user LLM decode is usually limited by memory bandwidth, not FLOPS.KV cache and contextHow longer context consumes memory and lowers decode speed.MoE vs dense modelsWhy MoE models use total parameters for fit and active parameters for speed.OffloadingWhat happens when part of a model spills from GPU memory into system RAM.Prefill vs decodeThe two different performance phases behind time to first token and generation speed.MTP and speculative decodingHow draft tokens accelerate decoding, and why speed multipliers do not stack freely.Choosing an inference stackA practical runtime choice based on hardware, model fit, and workload.Quantization formatsHow to choose among Q8, Q4, activation-aware, and native low-bit formats.Multi-GPU scalingHow memory pools grow and why interconnects decide the speed gain.Uncertainty and calibrationWhy estimates are ranges, how measured runs tune them, and what p10 to p90 means.The craft of llm.fitHow the interface keeps estimates specific, inspectable, and honest.