Okay, the breakdown *by-language* of Artificial Analysis's "Omniscience" hallucination/capability benchmark is absolutel...

Okay, the breakdown *by-language* of Artificial Analysis's "Omniscience" hallucination/capability benchmark is absolutely fascinating.I feel like this is some of the first concrete evidence I've seen of my longtime lean: that the *correctness* tooling we obsess over in PLT spaces is one of the greatest possible value-adds to the LLM era. (Just look at the Rust row, holy shit.)Choosing a good language basically shifts the quality of output by an entire cost-category / order-of-magnitude; compare the "R / Astra" cell vs. the "Rust/Luna" cell.https://artificialanalysis.ai/evaluations/omniscience?swe-deep-dive=omniscience-index-across-languages#software-engineering-deep-dive #AI #slop #LLM #typesystems #PLT

Read Original

Related

Mastodon discussion 38m ago

Профили критических сбоев ИИ-агентов с функцией вызова с реальным эффектом: DeepSeek V4 Pro и Gemini 3.8 FlashИсследован...

Профили критических сбоев ИИ-агентов с функцией вызова с реальным эффектом: DeepSeek V4 Pro и Gemini 3.8 FlashИсследование поведения ИИ-агентов с вызовом функций, имеющих реальный ...