Bits per weight
Bits per weight, or bpw, is how much storage each model parameter uses after compression. Lower bpw makes a model smaller and usually faster to read from memory, but quality damage becomes visible at aggressive settings. Q4_K_M is the practical default in the research data, while Q8_0 is near-lossless and formats below Q3 need a clear warning.
Formula
weight size (GB) = parameters (billions) × bpw ÷ 8
Worked example: 8B at Q4_K_M
The research anchor uses an 8B model at Q4_K_M, about 4.6 GB after format overhead. Its measured RTX 4090 decode range is 140 to 180 tokens per second.
8 × 4.5 ÷ 8 = 4.5 GB of raw weights artifact size ≈ 4.6 GB