A few weeks back I watched an agentic pipeline burn a month of budget in three days. The instinct everyone reached for w...

A few weeks back I watched an agentic pipeline burn a month of budget in three days. The instinct everyone reached for was the obvious one: "the frontier model is too expensive, switch to a smaller one." That's the wrong diagnosis, and it's worth writing down why.When we finally tagged every token to the (step, tool, model) tuple that spent it, the picture was uncomfortable. The expensive model wasn't the problem. The problem was that every step — the trivial classifier, the summary, the hard reasoning call — silently defaulted to it, and nobody could see which step was responsible. We'd been arguing about which model to buy when we should have been arguing about which step was allowed to call it. Cost in an agent isn't a number. It's a tree.In a single LLM call, cost is one line item. In an agentic loop it's a tree: each step calls a model, maybe a tool, maybe a retry, maybe a sub-step that calls again. The bill at the end is the sum over that whole tree.If you only ever see the total...

Read Original

Related