The moment you open up AI chat or an internal AI platform to the whole company via API, token costs become a line item. ...

The moment you open up AI chat or an internal AI platform to the whole company via API, token costs become a line item. And that line grows with headcount, and grows again every time someone reaches for a more capable model. A few hundred dollars a month gets waved through as "experimentation." Once you're into the thousands, then tens of thousands, someone upstairs starts asking questions.Team leads and managers will be expected to explain whether the results justified the spend. If I'm being honest, token costs are a rounding error next to salaries... but if you leave things wide open, the bill balloons alarmingly fast, so some kind of brake is needed. The problem is that the obvious brake — showing each person "$xx" next to every request — makes everyone timid. People start using the tool with one eye on the meter, and adoption stalls.So how do you keep a brake on things while still getting people to actually use it? I've been chewing on this recently, and here's where I landed. TL;...

Read Original

Related