973 articles tagged with Benchmark
The UNLV International Gaming Institute's (IGI) AI Research Hub (AiR Hub) in collaboration with KPMG LLP, the U.S. audit, tax and advisory firm, today released The State of AI in G...
I expanded the #Vera benchmark to six models across three providers (Anthropic, OpenAI, Moonshot). Kimi K2.5 got every Vera problem right while only managing 86% in Python and 91% ...
While Vision-Language Models (VLMs) have achieved remarkable progress in static visual understanding, their deployment in complex 3D embodied environments remains severely limited....
Text-to-Audio-Video (T2AV) generation is rapidly becoming a core interface for media creation, yet its evaluation remains fragmented. Existing benchmarks largely assess audio and v...
Organizational dynamics at Meta yielded a leaderboard for AI usage because it is a legible compromise for leadership. It turns the work into a tokenized game to force an "agentic" ...
I run an autonomous eval agent against new coding-agent stacks before trusting their numbers. The...
📰 GLM-5.1 on Huawei Cloud: The 2026 AI Benchmark with Open-Weight PerformanceGLM-5.1 has launched on Huawei Cloud, offering users access through multiple integrated products. The m...
📰 GLM-5.1 Outperforms Opus4.6: Open Source AI Breakthrough Sets New Benchmark (2026)The open source model GLM-5.1 has unexpectedly surpassed Opus4.6 in performance benchmarks, trig...
Recent advances in vision-language models (VLMs) have improved image captioning for cultural heritage. However, inferring structured cultural metadata (e.g., creator, origin, perio...
Gemma 4 dropped with a multi-token prediction (MTP) head and immediately every benchmark thread on...
Benchmark for evaluating color understanding in text-to-image models | CVPR 2026
Back in March I released agent-egress-bench, a test corpus for evaluating security tools that sit...
I just read that Meta has an internal leaderboard for LLM token usage, and I really can't imagine a less thoughtful, less useful status metric. Even assuming that the tokens are be...
A 600-run benchmark by Yusuke Endoh tested #ClaudeCode across 13 #ProgrammingLanguages by implementing a simplified Git.Key takeaways:⇨ Ruby, Python & JavaScript were the fastest, ...
Large language models are increasingly deployed as autonomous agents executing multi-step workflows in real-world software environments. However, existing agent benchmarks suffer f...
Large language models (LLMs) are widely explored for reasoning-intensive research tasks, yet resources for testing whether they can infer scientific conclusions from structured bio...
Microsoft udostępnia trzy autorskie modele MAI w ramach platformy Foundry. Wśród nowości znalazł się lider benchmarków transkrypcji oraz narzędzia do błyskawicznej syntezy mowy i f...
📰 LTX-Video 2.3 Benchmark: 11 Models Tested for AI Video GenerationLTX-Video 2.3 emerges as a leading AI video generation framework, with 11 distinct models tested across hardware ...
📰 Gemma 4 Leads 2026 AI Leaderboard with $0.20 Inference Cost and 100% Survival RateGemma 4, a 31-billion-parameter model, has shattered benchmarks by achieving 1,144% median ROI a...
📰 AI Memory Benchmark 2026: Chinese Teen Coders Break Records with 98.7% Referential Resolution Acc...A new AI memory benchmark, pioneered by teenage developers from China, has ach...
Is AI just a benchmark race? Google’s Gemini 3.1 Pro says yes, but 1 million medical images processed in Galicia say there’s a much deeper story. From doctor's offices to insurance...
Google DeepMind just scored 85% on ARC-AGI-2 — the most difficult general reasoning benchmark in AI.Previous best? 54%.This kind of leap doesn't happen often. General reasoning is ...
new writeup: why eval pipelines matter more than model selection.teams with good evals ship 3x faster. teams without them are flying blind, afraid to change anything because they c...