Tensor, pipeline, and data parallelism solve three different problems: latency, fit, and throughput. How each one splits an LLM across GPUs, what it costs in communication, and how production systems combine all three.
Tensor vs. Pipeline vs. Data Parallelism: How LLM Inference Scales Across GPUs
