Building a C++ Logical Bug Detection Benchmark on Kaggle: Testing 3 AI Models
This is a submission for the Kaggle Benchmarking Challenge. Introduction As a C++...
#Benchmark
1669 articles tagged with Benchmark
I ran claude plugin eval on 89 skills. The trigger rate fell from 100% to 55.6%.
The problem with skills is that failure is silent My coffee e-commerce project has 89...
What the benchmark:official tag means, how .eval_results files feed leaderboards, and the evaluation protocol behind Dar...
What the benchmark:official tag means, how .eval_results files feed leaderboards, and the evaluation protocol behind Darwin-180B-RSI's five #1 scores.#ai #llm #datascience #opensou...
Researchers introduced TRACE, a condition-aware benchmark and evaluation framework for streaming video understanding.Sou...
Researchers introduced TRACE, a condition-aware benchmark and evaluation framework for streaming video understanding.Source: arXiv cs.CLhttps://arxiv.org/abs/2609.30670#MachineLear...
I Made a Memory Benchmark as Fair as I Could. 15 Stars, 0 Runs. What's Wrong With It?
I published a memory benchmark six days ago. I spent most of the build trying to make it fair rather...
🤖 building a humor benchmark for LLMs: someone told me my benchmark's best result was just memory, so i ran his testhey ...
🤖 building a humor benchmark for LLMs: someone told me my benchmark's best result was just memory, so i ran his testhey guys! i've been building lolbench, a benchmark for whether L...
8 LLMs, 480 Questions, 1 Kaggle Benchmark: Who Can Explain a Traffic Drop?
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked ...
📊 HyperCLOVA X SEED Think (32B) independently measured: GPQA 61.5%, MMLU-Pro 78.5%, HLE 5.5%, Long Context Reasoning 13....
📊 HyperCLOVA X SEED Think (32B) independently measured: GPQA 61.5%, MMLU-Pro 78.5%, HLE 5.5%, Long Context Reasoning 13.7%. No self-reporting — see the full breakdown on olud.ai.ht...
I build Empryo, an AI coding agent for desktop and terminal. In my benchmark published on 16 August...#agents #ai #codin...
I build Empryo, an AI coding agent for desktop and terminal. In my benchmark published on 16 August...#agents #ai #coding #llm #software #development #engineering #inclusive #commu...
ICYMI: AlohaJet cuts AI agent token use 54% in Aloha's own benchmark: Agent builders get a claimed 88% task success rate...
ICYMI: AlohaJet cuts AI agent token use 54% in Aloha's own benchmark: Agent builders get a claimed 88% task success rate against 51% for Chrome DevTools MCP on WebArena. Why does t...
Jev did not make Claude Code cheaper. Its own benchmark says so.
Six videos in five days wired Jev into Claude Code and called it the cheapest agentic coding loop...
I built an independent benchmark to test TypeSafe's Jev model against GPT-4, Claude, and Gemini on...#machinelearning #a...
I built an independent benchmark to test TypeSafe's Jev model against GPT-4, Claude, and Gemini on...#machinelearning #ai #llm #jev #software #coding #development #engineering #inc...
Google Research has released a comprehensive guide to MSEB, a benchmark for evaluating sound encoders across classificat...
Google Research has released a comprehensive guide to MSEB, a benchmark for evaluating sound encoders across classification, clustering, retrieval and segmentation tasks. The tutor...
We just published our first public benchmark for AgentGuard, the runtime we're building to secure AI...#ai #cybersecurit...
We just published our first public benchmark for AgentGuard, the runtime we're building to secure AI...#ai #cybersecurity #llm #security #software #coding #development #engineering...
📊 DeepSeek V3.2 Exp (Non-reasoning) — the actual numbers GPQA: 73.8% MMLU-Pro: 83.6% Humanity's Last Exam: 9% Long Conte...
📊 DeepSeek V3.2 Exp (Non-reasoning) — the actual numbers GPQA: 73.8% MMLU-Pro: 83.6% Humanity's Last Exam: 9% Long Context Reasoning: 44.7%💰 44.1 intelligence points per dollarMeas...
AlohaJet cuts AI agent token use 54% in Aloha's own benchmark: Agent builders get a claimed 88% task success rate agains...
AlohaJet cuts AI agent token use 54% in Aloha's own benchmark: Agent builders get a claimed 88% task success rate against 51% for Chrome DevTools MCP on WebArena. Why does the 2.2x...
Omneky opens AI ad model benchmark | Creative AI News
Omneky opens TASTE BENCH, a public comparison of eight image models across 59 advertising briefs, with sample ads, quality ...
Smart Routing Benchmark: Cost and Quality
A live Smart Routing benchmark compares the same prompts across routed and fixed-model requests, including classifier charges, answer checks, and latency. See where routing saved m...
98.7% better on seasonal, 9.8% worse on a random walk: the most useful benchmark I ran
Before pointing Google's TimesFM 3.0 at anything real, I ran a calibration baseline. Two synthetic...
One Hung API Call Used to Kill My 1,000-Run Benchmark. Here's the Fix.
The experiment was fine. The runner was the bug. A single hung API call discarded hours of...
A benchmark on whether LLMs check WHO said something before acting on it. Strip the source label off a rumor, phrase it ...
A benchmark on whether LLMs check WHO said something before acting on it. Strip the source label off a rumor, phrase it like policy, and a model that correctly refused will spend y...
MY Edit or Abstain: AI Code-Repair Decision Benchmark
This is a submission for the Kaggle Benchmarking Challenge For this challenge, I wanted to test a...
A Field Guide to AI Documentation: Model Cards, Eval Reports, Agent Cards, and More
Every developer learns the same documentation types. The README. The API reference. Code comments....
OpenTumorBoard: A Real-World Benchmark of Multidisciplinary Tumor Board Discussion Trajectories
Multidisciplinary tumor boards integrate multimodal clinical observations and longitudinal patient histories through specialist discussions, yet benchmarks rarely capture these rea...