Multi-GPU scaling
Adding GPUs nearly adds their memory capacity, after reserving 1 to 2 GB per card, but decode speed depends on how the cards exchange work. NVLink tensor parallelism reaches about 0.8 times the GPU count in effective bandwidth. PCIe tensor parallelism reaches about 0.55 times the count. Pipeline parallelism is mainly a capacity move at batch one, with roughly single-GPU decode bandwidth.
Formula
effective decode BW ≈ single-GPU BW × factor NVLink factor = 0.8N; PCIe factor = 0.55N; pipeline factor = 1
Worked example: Two RTX 3090 cards over NVLink
Each RTX 3090 contributes 24 GB of VRAM and 936 GB/s of peak bandwidth. Two cards provide a 48 GB physical pool before per-card reserves, while the research scaling rule gives about 1,498 GB/s of effective decode bandwidth over NVLink.
memory pool = 2 × 24 = 48 GB before reserves effective BW = 936 × (2 × 0.8) ≈ 1,498 GB/s