Papers with Code paper Nov 2

The SIMPLE Law for Energy Cost of LLM Inference

Generative AI is billed per token, yet tokens are not equally expensive. Each token enters the KV-cache and raises the cost of those that follow, so energy grows non-linearly along...