Quantization formats

A format is an algorithm, not a bit count. It fixes how many weights share a scale, whether an offset is stored alongside it, and whether important weights are protected from rounding. Q8_0 is effectively lossless, Q4_K_M is the llama.cpp default at roughly 0.8 percent perplexity cost, and AWQ and GPTQ target serving stacks rather than local files. Formats below 3 bits only hold up on a large model or on weights trained at that precision from the start.