1661 articles tagged with Benchmark
SwarmReconGuard is a reproducible black-box benchmark that observes only service-boundary telemetry.Source: arXiv cs.MAhttps://arxiv.org/abs/2610.09138#MachineLearning
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked BankSafe-Bench: Can AI...
I build Operant, so I have a bias. This post first went out on our blog. I post it here because I...
I maintain django-explain-errors, a Django middleware that catches unhandled exceptions in...
We present Pumpire, a unified benchmark for evaluating metric point-pair distance estimation capability of both image- and video-level 3D foundation models, with or without depth p...
Le plus dur n'est pas d'ecrire un agent LLM, c'est de savoir quand il casse.Agent Eval Kit (19 USD) : 90 cas de test prets: 4 suites, runner multi-providers, score par run, seuil d...
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked An agent...
My Benchmark Caught Me Lying Before It Caught Any Model: Do Coding Agents Report "Verified" Honestly?#devchallenge #kagglechallenge #ai #machinelearning #software #coding #developm...
I wrote a task for Terminal-Bench, a public benchmark of hard command-line tasks for AI agents. The...#agents #pdf #python #ai #software #coding #development #engineering #inclusiv...
StarSkirmish Bench: a Benchmark where LLMs write Bots for StarCraft: Brood War VideoGame. Each LLM gets one hour to write a Protoss bot in C++, which is then evaluated against othe...
US stock market hits all-time high as investors continue to bet on AIThe benchmark S&P 500 is up 14 percent this year despite elevated oil prices and US government debt sell-off.ht...
My agent eval suddenly got much worse. The skill had barely changed, but the model behind the alias had.That led to a more uncomfortable discovery: sometimes the LLM judge was nois...
OMIT is a benchmark of 218 paired-frame scenarios across ten conflict types built from a five-perspective philosophical panel.Source: arXiv cs.CLhttps://arxiv.org/abs/2610.07847#Ma...
A mechanism explainer on LLM evaluation using the Team Recruitment Oracle benchmark: same prompts, a fixed judge, per-axis rubrics, and a public verbatim trail. Real scores for Nem...
This is a submission for the Kaggle Benchmarking Challenge A Kaggle benchmark about the most boring...#devchallenge #kagglechallenge #machinelearning #ai #software #coding #develop...
Dense visual text requires image generators to reproduce long strings across multiple regions with correct placement and legibility. As short-string rendering improves, evaluation ...
Product-centric advertisement video generation aims to create promotional videos that preserve fine-grained product identity while presenting selling points through coherent multi-...
Vision-and-Language Navigation (VLN) has largely focused on a single agent following a single instruction, yet many real-world applications require teams of robots to tackle tasks ...
AI agents: how can you assess reported speed gains and benchmark scores?https://podcast.aequitus.net/episodes/2026-10-04-saturday-ai-agents-how-can-you-assess-reported-speed-gain-e...
SGAnalog introduces an end-to-end circuit benchmark from open-source silicon tapeouts. It contains 273 topologically distinct top-level designs. Across seven models on 66 transcrip...
Granite 4.0 1B: GPQA 28.1%, MMLU-Pro 32.5%, HLE 4.8%, Long Context 6% — measured independently, not self-reported. For a 1B, that spread is the reality. Open bench page for full pi...
A build-in-public deep dive from the smallOps project Why this benchmark exists . smallOps...
Researchers measured how reference choice moves a benchmark score by holding the scored output fixed and exchanging the reference. The score moves by 4.95 F1 points for one output ...
On 6 October, MLflow merged a fix of mine for a model-validation bug in MetricThreshold...