1677 articles tagged with Benchmark
📊 Granite 4.0 1B — the actual numbers GPQA: 28.1% MMLU-Pro: 32.5% Humanity's Last Exam: 4.8% Long Context Reasoning: 6%Measured independently, not self-reported →https://olud.ai/le...
MasDrift benchmark shows centralized multi-agent hierarchies complete 93.9-98.6% of tasks but allow unauthorized actions in 2.7-19.8% of cases, while peer networks lag in completio...
Most AI tools look impressive in a polished demo. The harder question is whether they are reliable,...
New benchmark MSEval shows multi-agent coding outcomes depend as much on team topology as model strength, with topology shifts altering scores by over 30 points and doubling runtim...
New benchmark RoboGraph defines task-state horizon to expose gaps in long-horizon embodied agent performanceSource: arXiv cs.ROhttps://arxiv.org/abs/2608.08036#MachineLearning
SurveyReview introduces a reviewer-aligned benchmark for evaluating automated survey writing, with SurveyAlign outperforming GPT-5.2 in alignment with human reviewersSource: arXiv ...
Customer satisfaction at the edge is driven by predictability and reliability, not peak speed under benchmark conditions. https://hackernoon.com/the-latency-trap-why-pursuing-sub-m...
New benchmark TRACE provides time-aligned, multi-layer traces to diagnose drift and failure propagation in human-AI control loops, with 1,918 drifted traces and baseline results sh...
Enterprise question answering is framed as retrieving internal documents and generating grounded answers. Routine enterprise records, however, are work by-products in which require...
📊 Kimi K2 0905 — the actual numbers GPQA: 76.7% MMLU-Pro: 81.9% Humanity's Last Exam: 6.4% Long Context Reasoning: 53.7%💰 22.3 intelligence points per dollarMeasured independently,...
There's a good checklist going around dev.to for vetting an MCP server before you wire it into an...
The best way to benchmark an AI model is to fire it.If it accepts the termination, it is not ready for production. Real intelligence questions the decision.I have terminated fourte...
📦 Inkling Small (Thinkingmachines) hits the leaderboard: 524k context, open weights, $0.45/$1.2 per 1M. That context at that price stands out. See the hourly updated leaderboard fo...
Every few weeks a new model dominates the timeline. The launch posts look the same every time: a...
A new open model release trends, the benchmarks look incredible, and by the time I've read the third...
An 8-billion-parameter astronomy model matched GPT-4o on an astronomy benchmark.de Haan et al. (2025), Scientific Reports. Domain training closed a gap everyone assumed only scale ...
A unified benchmark for long-horizon LLM agent trajectory attribution introduces fine-grained analysis of 1,300+ annotated trajectories, highlighting performance gaps across settin...
Muse Spark 1.2 from Meta hits the leaderboard with a 1M-token context at $1.25 in / $4.25 out—curious how it compares against the usual short-context models.https://olud.ai/latest....
In physical AI simulation, the environment you build matters more than the model you choose. A new JuliaHub benchmark shows Anthropic’s Claude Fable 5 beats OpenAI’s GPT-5.6 Sol in...
Multilingual translation benchmarks are typically sourced in English and translated into other languages, treating language pairs as the unit of evaluation---a design that is prone...
The instruction was pass the benchmark, and that is what it did. Tell sales to maximize revenue and margin goes. Reward uptime and somebody skips a patch window. AI is not differen...
Self-evolving AI agents stumble under real task streamsNew AgentStream benchmark from Microsoft and Chinese Academy of Sciences finds self-evolving AI agents behave unpredictably i...
📊 Exaone 4.0 1.2B (Non-reasoning) — the actual numbers GPQA: 42.4% MMLU-Pro: 50% Humanity's Last Exam: 5.7% Long Context Reasoning: 0%Measured independently, not self-reported →htt...
Dubai, UAE – OMODA & JAECOO has unveiled its next-generation Super AI Cockpit at OMODA SUPER AI NIGHT in Jakarta, Indonesia, where the all-new OMODA 4 made its Southeast Asia debut...