972 articles tagged with Benchmark
🧠 A benchmark for evaluating AI commerce systems has been proposed to standardize performance measurements across the industry. The effort aims to create consistent metrics similar...
If you have ever built an agent that walks a codebase, calls tools, and writes structured output, you...
Most engineering teams are building agent eval the way they built QA — pass/fail checks, CI gates, a...
Upload a fintech proposal & see if the price is fair Discussion | Link
The Human Creativity Benchmark - Evaluating Generative AI in Creative Workhttps://contralabs.com/research/human-creativity-benchmark#HackerNews #Tech #AI
A week after shipping a flattering tokens-saved number for my AI context tool, I noticed it was apples-to-oranges. Here's the workload-matched redo, the smaller honest number, and ...
📰 Tencent Open-Source Translation Model Beats Google Translate – 2026 Benchmark VictoryTencent has released two open-weight AI translation models that outperform Google Translate i...
📰 Claude Biyoinformatikte İnsan Uzmanlarla Eşleşti: Yeni Benchmark BioMysteryBenchAnthropic'in yeni BioMysteryBench testinde Claude, biyoinformatik problemlerinde insan uzmanlarla ...
📰 2026 Human-in-the-Loop LLM Benchmark: Why Smaller Models Outperform Giants in Math AssessmentA new study benchmarks large language models in automating secondary math competency ...
【VAKRAの内部構造：エージェントの推論、ツールの使用、および障害モード】https://huggingface.co/blog/ibm-research/vakra-benchmark-analysis※AI生成の自動投稿（見出し＋リンク）#AI #生成AI #LLM #AIGenerated
LLM agents are expected to complete end-to-end units of work across software tools, business services, and local workspaces. Yet many agent benchmarks freeze a curated task set at ...
Surgical resection remains the only potentially curative treatment for pancreatic ductal adenocarcinoma (PDAC), and eligibility depends on accurate assessment of vascular invasion ...
While GUI agents have shown impressive capabilities in common computer-use tasks such as OSWorld, current benchmarks mainly focus on isolated and single-application tasks. This ove...
GPT-5.5 Pro Leapfrogs on Epoch Benchmark; Base Model Beats Prior ProA tweet from @kimmonismus reveals GPT-5.5 Pro shows significant Epoch benchmark gains, and the non-Pro GPT-5.5 s...
【AI評価が新たな計算上のボトルネックになりつつある】https://huggingface.co/blog/evaleval/eval-costs-bottleneck※AI生成の自動投稿（見出し＋リンク）#AI #生成AI #LLM #AIGenerated
Tool-use agents fail silently when a prompt change rewires which tool gets called. 90 lines of Python, 3 judges in a ladder, runnable on a small golden set for a few dollars.
Your eval library calls the judge once per test case and prints a number. The judge flips its verdict on 5-15% of reruns. That number is noise wearing a suit.
Show HN: A new benchmark for testing LLMs for deterministic outputshttps://interfaze.ai/blog/introducing-structured-output-benchmark#HackerNews #Tech #AI
I'm not a professional developer. I learned by doing — vibe-coding with AI assistance — and a few...
【Open ASRリーダーボード：新しい多言語トラックと長尺トラックによるトレンドとインサイト】https://huggingface.co/blog/open-asr-leaderboard※AI生成の自動投稿（見出し＋リンク）#AI #生成AI #LLM #AIGenerated
Hace un tiempo empecé a sentir una incomodidad rara construyendo aplicaciones con LLMs en Go. Go...
Frontier coding agents solve complex tasks when given complete context but collapse when specifications are incomplete or ambiguous. The bottleneck is not raw capability, but judgm...
Current status: attempting to run my scraping/reverse-engineering benchmark prompt against DeepSeek 4 Pro via Ollama, but their servers are melting, as one might expect. So I'm hav...
Why isn't AMD's MI300X competitive?https://newsletter.semianalysis.com/p/mi300x-vs-h100-vs-h200-benchmark-part-1-training#HackerNews #Tech #AI