What Inference-Platform Benchmark Posts Leave Out
DCGM stops at host-level GPU counters. Kernel-side eBPF adds the per-rank, per-tenant signals...
#Benchmark
1697 articles tagged with Benchmark
【Open ASRリーダーボード：新しい多言語トラックと長尺トラックによるトレンドとインサイト】https://huggingface.co/blog/open-asr-leaderboard※AI生成の自動投稿（見出し＋リンク）#AI #...
【Open ASRリーダーボード：新しい多言語トラックと長尺トラックによるトレンドとインサイト】https://huggingface.co/blog/open-asr-leaderboard※AI生成の自動投稿（見出し＋リンク）#AI #生成AI #LLM #AIGenerated
I Opened an AI Coding Challenge “Just to Look.” A Few Hours Later, I Was Completely Invested in the Leaderboard.
What started as a simple AI-generated ice cream game somehow turned into one of the most...
An empirical benchmark of three frontier LLMs on the SmartBugs dataset, with one methodology gotcha that almost cost a m...
An empirical benchmark of three frontier LLMs on the SmartBugs dataset, with one methodology gotcha that almost cost a model 20 points of measured recall. https://hackernoon.com/ca...
Edit-Compass & EditReward-Compass: A Unified Benchmark for Image Editing and Reward Modeling
Recent image editing models have achieved remarkable progress in instruction following, multimodal understanding, and complex visual editing. However, existing benchmarks often fai...
Vividh-ASR: A Complexity-Tiered Benchmark and Optimization Dynamics for Robust Indic Speech Recognition
Fine-tuning multilingual ASR models like Whisper for low-resource languages often improves read speech but degrades spontaneous audio performance, a phenomenon we term studio-bias....
RealICU: Do LLM Agents Understand Long-Context ICU Data? A Benchmark Beyond Behavior Imitation
Intensive care units (ICU) generate long, dense and evolving streams of clinical information, where physicians must repeatedly reassess patient states under time pressure, undersco...
CurveBench: A Benchmark for Exact Topological Reasoning over Nested Jordan Curves
We introduce CurveBench, a benchmark for hierarchical topological reasoning from visual input. CurveBench consists of 756 images of pairwise non-intersecting Jordan curves across e...
Benchmark Results: SmolLM3 3B, Phi-4-mini, DeepSeek V4, Grok 4.20 — Agent Coding Tested
The second round of the Works With Agents agent coding benchmark is in — 32 models tested this time,...
I built the first open benchmark for federal contracting AI. Here's what it shows about frontier LLMs.
Frontier LLMs hallucinate FAR clause numbers somewhere between 0% and 32% of the time. A specialized 150M-parameter model trained in 4 minutes matches Claude Haiku on F1 with less ...
OpenAI Codex vs Claude Code: Hands-On Python Benchmark for Devs
We pointed Codex and Claude Code at the same Python codebase across refactoring, debugging, and agentic tasks. Here is what each tool shipped, where each one wins, and what the spe...
Claude as a User-Space IP Stack: What an ICMP Ping Benchmark Reveals About LLM Latency
Adam Dunkels wired Claude into a user-space TCP/IP stack and benchmarked it against ICMP ping. The latency floor it reveals is the most honest stress test we have for agentic Claud...
The Benchmark Leading Tech PR Firm: Omri Hurwitz Media
The technology public relations industry is undergoing a structural evolution as startups compete in an environment where attention is fragmented across countless digital channels....
WildRelight: A Real-World Benchmark and Physics-Guided Adaptation for Single-Image Relighting
Recent single-image relighting methods, powered by advanced generative models, have achieved impressive photorealism on synthetic benchmarks. However, their effectiveness in the co...
Covering Human Action Space for Computer Use: Data Synthesis and Benchmark
Computer-use agents (CUAs) automate on-screen work, as illustrated by GPT-5.4 and Claude. Yet their reliability on complex, low-frequency interactions is still poor, limiting user ...
Visual Aesthetic Benchmark: Can Frontier Models Judge Beauty?
Multimodal large language models (MLLMs) are now routinely deployed for visual understanding, generation, and curation. A substantial fraction of these applications require an expl...
ShapeCodeBench: A Renewable Benchmark for Perception-to-Program Reconstruction of Synthetic Shape Scenes
We introduce ShapeCodeBench, a synthetic benchmark for perception-to-program reconstruction: given a rendered raster image, a model must emit an executable drawing program that a d...
PreScam: A Benchmark for Predicting Scam Progression from Early Conversations
Conversational scams, such as romance and investment scams, are emerging as a major form of online fraud. Unlike one-shot scam lures such as fake lottery or unpaid toll messages, t...
A new Microsoft Research benchmark called DELEGATE-52 found something enterprise teams need to know: even the best model...
A new Microsoft Research benchmark called DELEGATE-52 found something enterprise teams need to know: even the best models (Gemini 3.1 Pro, Claude 4.6 Opus, GPT 5.4) corrupted 25% o...
STRADVISION Sets New Global Benchmark for ADAS with Launch of Production-Ready Integrated Perception Platform
Strategic collaboration with leading Physical AI developer delivers a turnkey solution for accelerated Level 2+ vehicle integration SEOUL, South Korea, May 12, 2026 /PRNewswire/ --...
🧠 This article ranks and reviews artificial intelligence tools designed for social media use in 2026. The tools are eval...
🧠 This article ranks and reviews artificial intelligence tools designed for social media use in 2026. The tools are evaluated across various categories to help users select options...
oh-my-agent: 9 new skills, cursor as first-class vendor, 80/100 benchmark
When you tell an agent to scaffold a Next.js app, it picks the wrong version, ignores your lint...
Agentick Benchmark: GPT-5 Mini Tops at 0.309, No Agent Paradigm DominatesAgentick benchmark evaluates RL, LLM, VLM, and ...
Agentick Benchmark: GPT-5 Mini Tops at 0.309, No Agent Paradigm DominatesAgentick benchmark evaluates RL, LLM, VLM, and hybrid agents on 37 tasks. GPT-5 mini leads at 0.309 ONS, bu...
RoboMemArena: A Comprehensive and Challenging Robotic Memory Benchmark
Memory is a critical component of robotic intelligence, as robots must rely on past observations and actions to accomplish long-horizon tasks in partially observable environments. ...