1676 articles tagged with Benchmark
Researchers present the Era by Eon Benchmark to evaluate LLM agents for enterprise systems using a generated fictional company with exact ground truth.Source: arXiv cs.AIhttps://ar...
🤖 【Hugging Face Papers】DF26: We Cannot Tell Fake From Real AnymoreWe introduce DF26, a novel benchmark for detecting AI-generated videos containing fully synthetic clips produced b...
"⚖️ Open vshttps://olud.ai/leaderboard.html#OpenSource #AI #LLM
Benchmark researchers and developers of large language models (LLMs) and other AI systems need to find relevant evaluations, locate their benchmark datasets and code, and understan...
📦 Nex-N2.5-Mini (free) hits the leaderboard with 262.1k tokens of context, open weights and free tier. A rare combination. Find it on the hub.https://olud.ai/latest.html#AI #LLM #O...
RT @pilvar222: Wir haben DeepSeek V4.1 Flash auf unserem Cybersicherheits-Benchmark getestet. Die Leistung ist jetzt deutlich besser. mehr auf Arint.info #AI #Benchmark #CVE #Cyber...
Disclosure first: I write digline, a small Python library for regression testing LLM applications....
📊 Independently measured, Seed-OSS-36B-Instruct delivers 40.3 intelligence points per dollar — GPQA 72.6%, MMLU-Pro 81.5%. Cost-efficiency that puts bigger models on notice.https:/...
Researchers introduced CLLPU to benchmark propagation boundaries in multilingual LLM unlearning.Source: arXiv cs.CLhttps://arxiv.org/abs/2609.05976#MachineLearning
SinoGlyphBench is a diagnostic benchmark that evaluates Chinese glyph-level obfuscation in language-model moderation.Source: arXiv cs.CLhttps://arxiv.org/abs/2609.05843#MachineLear...
Ran Qwen 3.8 27B 4-bit quantization benchmarks on RTX 4090 and M-series Mac for local AI agents. Don't fall for 1-bit — here's why.
The MERIT benchmark measures how long-term memory affects tool-using LLM agents under explicit cost accounting.Source: arXiv cs.AIhttps://arxiv.org/abs/2609.05441#MachineLearning
📦 Fresh on the leaderboard: Mercury 2.5 (Inception)Context: 260k tokens · $0.04 in / $0.15 out per 1M · commercialAll the latest models, tracked hourly:https://olud.ai/latest.html#...
The ABLE benchmark evaluates large language model agents on protein design tasks using tools such as ProteinMPNN and AlphaFold3.Source: arXiv cs.AIhttps://arxiv.org/abs/2609.05818#...
#LLM benchmark 就像二次元观众的萝莉妈妈一样，出现的时候就已经太老了。被模型特调是注定的结局。
Reacting to sudden physical hazards (catching a slipping plate, dodging a falling knife) is both a meaningful test of embodied intelligence and a hard requirement for deploying mul...
community llm benchmark questions ( brainstorm ) continual development https://lemmy.ml/post/52473722
📊 Devstral Small (May '25) — the actual numbers GPQA: 43.4% MMLU-Pro: 63.2% Humanity's Last Exam: 4% Long Context Reasoning: 32%Measured independently, not self-reported →https://o...
Every number in this article comes from reproducible evaluation scripts. Measurement basis: LoCoMo...
Welcome to the final entry of our series about designing, analyzing and visualizing AI evals! As with...
SWE-Bench Pro has emerged as a standard benchmark for evaluating software engineering agents on challenging repository-level tasks. However, our analysis work show that its evaluat...
Leading benchmarks for formal theorem proving with large language models are small collections drawn from competition math, such as the IMO and Putnam, that poorly represent field-...
When Execution Becomes Abundant. GPT-6 Astra’s 62.7% and 99.9% benchmark results show that capability depends on the system around the model. I trace what that means for enterprise...
Grok 4.6 Scores 26% and 88% on the Same Benchmark LinexAI's own model card reports Terminal-Bench 3.0 at 26.0%. Artificial Analysis reports 88.4% on 2.1. Both are the same model. T...