1697 articles tagged with Benchmark
📰 China's AI Advancement in 2026: DeepSeek Shatters US Benchmark ClaimsChina's AI progress, led by DeepSeek, contradicts recent US government claims of an eight-month lag. Governme...
I Built a Benchmark for the Failures Generic LLM Evaluations Miss Generic LLM benchmarks...
This post documents a real negative result: my trained model worked… but a well-written prompt worked...
General sales benchmarks often miss how real outbound agents fail: overclaiming on weak signals,...
The rapid advancement of Multimodal Large Language Models (MLLMs) has empowered Unmanned Aerial Vehicle (UAV) with exceptional capabilities in spatial reasoning, semantic understan...
🧠 A benchmark for evaluating AI commerce systems has been proposed to standardize performance measurements across the industry. The effort aims to create consistent metrics similar...
If you have ever built an agent that walks a codebase, calls tools, and writes structured output, you...
Most engineering teams are building agent eval the way they built QA — pass/fail checks, CI gates, a...
Upload a fintech proposal & see if the price is fair Discussion | Link
The Human Creativity Benchmark - Evaluating Generative AI in Creative Workhttps://contralabs.com/research/human-creativity-benchmark#HackerNews #Tech #AI
A week after shipping a flattering tokens-saved number for my AI context tool, I noticed it was apples-to-oranges. Here's the workload-matched redo, the smaller honest number, and ...
📰 Tencent Open-Source Translation Model Beats Google Translate – 2026 Benchmark VictoryTencent has released two open-weight AI translation models that outperform Google Translate i...
📰 Claude Biyoinformatikte İnsan Uzmanlarla Eşleşti: Yeni Benchmark BioMysteryBenchAnthropic'in yeni BioMysteryBench testinde Claude, biyoinformatik problemlerinde insan uzmanlarla ...
📰 2026 Human-in-the-Loop LLM Benchmark: Why Smaller Models Outperform Giants in Math AssessmentA new study benchmarks large language models in automating secondary math competency ...
【VAKRAの内部構造：エージェントの推論、ツールの使用、および障害モード】https://huggingface.co/blog/ibm-research/vakra-benchmark-analysis※AI生成の自動投稿（見出し＋リンク）#AI #生成AI #LLM #AIGenerated
LLM agents are expected to complete end-to-end units of work across software tools, business services, and local workspaces. Yet many agent benchmarks freeze a curated task set at ...
Surgical resection remains the only potentially curative treatment for pancreatic ductal adenocarcinoma (PDAC), and eligibility depends on accurate assessment of vascular invasion ...
While GUI agents have shown impressive capabilities in common computer-use tasks such as OSWorld, current benchmarks mainly focus on isolated and single-application tasks. This ove...
GPT-5.5 Pro Leapfrogs on Epoch Benchmark; Base Model Beats Prior ProA tweet from @kimmonismus reveals GPT-5.5 Pro shows significant Epoch benchmark gains, and the non-Pro GPT-5.5 s...
【AI評価が新たな計算上のボトルネックになりつつある】https://huggingface.co/blog/evaleval/eval-costs-bottleneck※AI生成の自動投稿（見出し＋リンク）#AI #生成AI #LLM #AIGenerated
Tool-use agents fail silently when a prompt change rewires which tool gets called. 90 lines of Python, 3 judges in a ladder, runnable on a small golden set for a few dollars.
Your eval library calls the judge once per test case and prints a number. The judge flips its verdict on 5-15% of reruns. That number is noise wearing a suit.
Show HN: A new benchmark for testing LLMs for deterministic outputshttps://interfaze.ai/blog/introducing-structured-output-benchmark#HackerNews #Tech #AI
I'm not a professional developer. I learned by doing — vibe-coding with AI assistance — and a few...