Banking intents, 150 public examples, 10 models: GPT-6.1 Terra is the cheapest shown to match GPT-6.1 Sol (84.0% vs 86.0...

Banking intents, 150 public examples, 10 models: GPT-6.1 Terra is the cheapest shown to match GPT-6.1 Sol (84.0% vs 86.0%, within a 5-point margin) at $0.000727 per successful task, 42.8% less. Retries and failed attempts are counted in every price. Table, failures, re-run script: https://clearbar.grain64.com/verdicts/banking77-intent/?utm_source=mastodon&utm_campaign=social-weekly #LLM #evals

Read Original

Related