Scale and zero point

Quantization maps a real range onto a handful of integer buckets using two constants: a scale, the ratio of the two ranges' widths, and a zero point, the bucket real zero lands in. Finding the range is calibration, trivial for weights and awkward for activations, which have to be watched on sample traffic. Dequantization reverses the map but cannot undo the rounding, and that residue is the entire quality cost. Everything else, block sizes, codebooks, importance weighting, is an argument about how to spend fewer bits per weight without enlarging it.

Formula

s = \frac{β - α}{q _{max} - q _{min}} q = round (\frac{r}{s}) + z r^= (q - z) s

Worked example: One weight into 4 bits

A weight of 0.42 in a layer spanning −1 to 1, mapped to a symmetric 4-bit range. Rounding to the nearest bucket costs 0.0086, and dequantization cannot recover it. Every quality argument about quantization is an argument about this residue, summed over billions of weights.

s = \frac{1 - ( - 1 )}{7 - ( - 7 )} = \frac{2}{14} = 0.142857 q = round (\frac{0.42}{0.142857}) = round (2.94) = 3 r^= 3 \times 0.142857 = 0.428571 ε = 0.428571 - 0.42 = 0.008571

See what 4-bit weights do to a 27B model