1676 articles tagged with Benchmark
🧪 LLM Benchmark Showdown: 5 lokale Ollama-Modelle im VergleichGetestet auf derselben Hardware (#gmktecevo2 #AMDRyzenAIMaxPlus395 #strixhalo):• #GSM8K (100 Samples) — Math• #BFCL (1...
Last spring, a colleague's team shipped an AI-generated query that accidentally exposed row-level...
Qwen 3.8 abliterated: Was bleibt von den Guardrails?Qwen 3.8 27B verweigert im AgentHarm-Benchmark 75% der schädlichen Aufgaben. Das abliterated Blackfrost-Paket erreicht 0% Verwei...
Four harnesses took the same public ARC-AGI-3 set from 13% to 100% without touching a single weight. Then Microsoft put the harness inside the training loop.
Fine-tuning NuExtract3 (LoRA) on 624 train / 44 eval books from this pilot corpus markedly improved precision (0.61→0.76), with recall essentially unchanged (0.76→0.75) vs. the zer...
🤖 KI-Briefing — 24.08.2026 1. Welche KI ist die beste? KI-Leaderboard (24.08.2026)Welche KI ist gerade die beste für Texte, Programmieren, Recherche oder Bilder? Täglich aktualisie...
DreamBench-SWE is a multi-session benchmark for software-agent memory hygiene. Later tasks depend on non-inferable evidence from earlier sessions.Source: arXiv cs.AIhttps://arxiv.o...
📊 Qwen3 4B 2507 (Reasoning) scores 66.7% GPQA and 74.3% MMLU-Pro, but only 6.2% on Humanity's Last Exam — independent measurements reveal the gaps. Full data here.https://olud.ai/l...
Industrial technical reports contain high-value knowledge for maintenance, troubleshooting, and product engineering, but their heterogeneous structure (dense prose, specifications,...
Last night, my AI partner and I built a memory system from scratch, tested it about forty different...
Ox Alpha's broader benchmark score dropped to 63% on a full 113-task test versus 80% on a smaller subset, raising questions about actual capability versus initial claims. Performan...
【VAKRAの内部構造：エージェントの推論、ツールの使用、および障害モード】https://huggingface.co/blog/ibm-research/vakra-benchmark-analysis※AI生成の自動投稿（見出し＋リンク）#AI #生成AI #LLM #AIGenerated
Or: "🎨 Qwen-Image-3.0-Pro trails Reve 2.1 byhttps://olud.ai/media-leaderboard.html#OpenSource #GenerativeAI #StableDiffusion #AI
📊 Qwen3 VL 4B (Reasoning) — the actual numbers GPQA: 49.4% MMLU-Pro: 70% Humanity's Last Exam: 4.6% Long Context Reasoning: 23%Measured independently, not self-reported →https://ol...
The self improving AI story keeps getting told, so it's worth reading a benchmark that tests it directly. Agents got 4 hours to rewrite a training algorithm for real gains. Result:...
Free AI coding credits are a measurement instrument, and most developers waste them by treating the...
A couple of day ago I posted about OmnisBench, our open benchmark for LLM routing, specifically our...
Week 34/52Replaybook became a real multi-harness benchmark with 240 comparable infrastructure trials. Agents of Empires got maps, fog of war, and evidence-driven replays. I built a...
Krasyn Note Check run over 36 transcript and note pairs from the MIT-licensed Omi Health medical-note-eval corpus: counts, examples, and the defects it exposed in our own tool. No ...
I ran the $0.14 model against the $0.44 model expecting a close fight. It lost 4-0. Then I looked at what it does to everything priced in between.
SOP-Bench is a new benchmark for evaluating AI agents on real business procedures. It pairs genuine enterprise SOPs with functioning tools and ground-truth answers.Source: Amazon S...
【音声認識におけるベンチマーク最適化の測定】https://huggingface.co/blog/asr-benchmark-optimization※AI生成の自動投稿（見出し＋リンク）#AI #生成AI #LLM #AIGenerated
【FFASRリーダーボードのご紹介：実世界におけるASRのベンチマーク】https://huggingface.co/blog/ffasr-leaderboard※AI生成の自動投稿（見出し＋リンク）#AI #生成AI #LLM #AIGenerated
We introduce ClosureBench, a constructive benchmark for compositional graph-relational reasoning with programmatically verified ground truth. Unlike fixed-test-set benchmarks vulne...