926 articles tagged with Benchmark
VibeLifeBench introduces a benchmark of 200 long-horizon tasks to test if LLM agents can act proactively and persistently in a changing simulated world. Seven leading models all sc...
New benchmark finds automated evaluation of tool-using LLM agents often unreliable; GPT-4o-mini surpasses heuristic judging, and runtime interceptors cut hallucinations by 24 perce...
RealReplicaBench offers developers a new tool for benchmarking agents in high-fidelity replicas of real online services, highlighting critical tradeoffs in AI training.
Lookspan's eval feature runs the first 100 items of a dataset. The panel said 'runs up to 100 items synchronously' — the same sentence whether your dataset had five items or five h...
So our last training pass on Qwen3-Omni-30B-A3B-Instruct — fine-tuning it on Barbados newspapers —...
📊 GLM-4.6 (Reasoning) — the actual numbers GPQA: 78% MMLU-Pro: 82.9% Humanity's Last Exam: 14.5% Long Context Reasoning: 55.3%💰 30.4 intelligence points per dollarMeasured independ...
When your benchmark is wrong and your model is right We fine-tuned a 30-billion-parameter model on...
This launch addresses a major gap felt in the Indian AI ecosystem, especially voice models. AI voice tools have made rapid advancements in Western languages, but diverse, multi-spe...
New TAF-MED benchmark shows 71.6% of LLM medical conversations contain unsafe responses, with 61.4% of initially safe interactions later collapsing to unsafe. Conversational safety...
Adversarial A/B testing of 13 AI coding models with keelwright safety skill. KDS scores: from 83 (Laguna S 2.1) to 0 (weak models that fabricate results).
👉 "PT2PR: A Benchmark for Multimodal Patent-to-Product Retrieval" by Lia Shahnazaryan & Stefan Heindorf, also at #CIKM2026 👏 Congratulations to all authors — in bocca al lupo! 🐺(th...
Xiaomi's MiLM Plus has released PROVE, a new perception-aligned benchmark for video object removal. The metrics RC-S and RC-T evaluate spatial coherence and temporal consistency wi...
In Hong Kong, HKU introduces RoboDojo, a unified benchmark for embodied AI to standardise robot reliability testingSource: University of Hong Kong Presshttp://www.hku.hk/press/news...
Qwen3.8 Max trails Claude Fable 5 by just 4 points on today's benchmark, yet costs 8x less per 1M output tokens. That gap is the whole story for budget-conscious builders—see where...
Our CEO Rob Imbeault published a piece on LinkedIn this week about a result our team posted: 99.95%...
Agentic systems powered by large language models (LLMs) have opened new opportunities for business ideation. Yet existing approaches remain confined to a text-only paradigm, despit...
I fine-tuned Mistral 7B on my laptop to detect personal data in log lines and support messages. On my...
Part 3 of a series building a support-ticket agent with no framework. Previous: Part 2 (tool...
Score your resume against any number of jobs locally — free, explainable, CPU or GPU. Powered by a 156M ModernBERT cross-encoder.
📊 Granite 4.0 1B — the actual numbers GPQA: 28.1% MMLU-Pro: 32.5% Humanity's Last Exam: 4.8% Long Context Reasoning: 6%Measured independently, not self-reported →https://olud.ai/le...
MasDrift benchmark shows centralized multi-agent hierarchies complete 93.9-98.6% of tasks but allow unauthorized actions in 2.7-19.8% of cases, while peer networks lag in completio...
Most AI tools look impressive in a polished demo. The harder question is whether they are reliable,...
New benchmark MSEval shows multi-agent coding outcomes depend as much on team topology as model strength, with topology shifts altering scores by over 30 points and doubling runtim...
New benchmark RoboGraph defines task-state horizon to expose gaps in long-horizon embodied agent performanceSource: arXiv cs.ROhttps://arxiv.org/abs/2608.08036#MachineLearning