Measure when quantization helps: latency, throughput and quality

Compare a 16-bit model with a weight-only quantized version at small and large batch sizes and on a small task set, and explain where speedups appear or vanish.

The exercise happens on Self-directed (serving engine with quantization support). Canonical link: https://docs.vllm.ai/en/latest/features/quantization/. Link check: link responded (5 Oct 2026). Opened from this app 0 times; opening is not counted as an attempt.

Supports: Needs a GPU and a quantization method supported on it. Prepares “when does quantization not help?” questions.