GDDR7 vs HBM3E for Inference: Bandwidth per Dollar Stops Being the Question

GDDR7 delivers 1.5 TB/s at a third of the cost. For batch-one serving that is enough, and the reason most inference clusters still buy HBM is not bandwidth at all.

Read Original

Related