llm-d 1.0 routes by prefix-cache affinity until a token-load limit. Break-even math, a 10-endpoint simulation, and how to calibrate your own limit.
When Should an LLM Router Stop Being Sticky? llm-d 1.0 Cache-Affinity Routing, Break-Even Math, and a Simulation
