1677 articles tagged with Benchmark
Humanoid motion tracking is central to teleoperation and whole-body imitation, yet evaluation often disagrees with what people perceive in videos. Kinematic errors average per-fram...
Long-form video understanding encompasses tasks that go beyond retrieving isolated events, including tracking an evolving narrative and interpreting social meaning that may remain ...
Last month I wrote about the six questions I ask every new model. Those questions are good for a...
Grok 4.6 matches GPT-5.6 Sol's reasoning benchmark at one-fifth the output cost, per independent testing. For agents running many prompts before completion, the pricing gap compoun...
VibeLifeBench introduces a benchmark of 200 long-horizon tasks to test if LLM agents can act proactively and persistently in a changing simulated world. Seven leading models all sc...
New benchmark finds automated evaluation of tool-using LLM agents often unreliable; GPT-4o-mini surpasses heuristic judging, and runtime interceptors cut hallucinations by 24 perce...
RealReplicaBench offers developers a new tool for benchmarking agents in high-fidelity replicas of real online services, highlighting critical tradeoffs in AI training.
Lookspan's eval feature runs the first 100 items of a dataset. The panel said 'runs up to 100 items synchronously' — the same sentence whether your dataset had five items or five h...
So our last training pass on Qwen3-Omni-30B-A3B-Instruct — fine-tuning it on Barbados newspapers —...
📊 GLM-4.6 (Reasoning) — the actual numbers GPQA: 78% MMLU-Pro: 82.9% Humanity's Last Exam: 14.5% Long Context Reasoning: 55.3%💰 30.4 intelligence points per dollarMeasured independ...
When your benchmark is wrong and your model is right We fine-tuned a 30-billion-parameter model on...
This launch addresses a major gap felt in the Indian AI ecosystem, especially voice models. AI voice tools have made rapid advancements in Western languages, but diverse, multi-spe...
New TAF-MED benchmark shows 71.6% of LLM medical conversations contain unsafe responses, with 61.4% of initially safe interactions later collapsing to unsafe. Conversational safety...
Adversarial A/B testing of 13 AI coding models with keelwright safety skill. KDS scores: from 83 (Laguna S 2.1) to 0 (weak models that fabricate results).
👉 "PT2PR: A Benchmark for Multimodal Patent-to-Product Retrieval" by Lia Shahnazaryan & Stefan Heindorf, also at #CIKM2026 👏 Congratulations to all authors — in bocca al lupo! 🐺(th...
Xiaomi's MiLM Plus has released PROVE, a new perception-aligned benchmark for video object removal. The metrics RC-S and RC-T evaluate spatial coherence and temporal consistency wi...
In Hong Kong, HKU introduces RoboDojo, a unified benchmark for embodied AI to standardise robot reliability testingSource: University of Hong Kong Presshttp://www.hku.hk/press/news...
Qwen3.8 Max trails Claude Fable 5 by just 4 points on today's benchmark, yet costs 8x less per 1M output tokens. That gap is the whole story for budget-conscious builders—see where...
Our CEO Rob Imbeault published a piece on LinkedIn this week about a result our team posted: 99.95%...
Agentic systems powered by large language models (LLMs) have opened new opportunities for business ideation. Yet existing approaches remain confined to a text-only paradigm, despit...
Video understanding is a fundamental task for evaluating the capabilities of multimodal large language models (MLLMs). However, existing leading models have already achieved approx...
I fine-tuned Mistral 7B on my laptop to detect personal data in log lines and support messages. On my...
Part 3 of a series building a support-ticket agent with no framework. Previous: Part 2 (tool...
Score your resume against any number of jobs locally — free, explainable, CPU or GPU. Powered by a 156M ModernBERT cross-encoder.