327 articles tagged with Benchmark
Cell-type-specific marker genes are fundamental to plant biology, yet existing resources primarily rely on curated databases or high-throughput studies without explicitly modeling ...
Autonomous agents have rapidly matured as task executors and seen widespread deployment via harnesses such as OpenClaw. Safety concerns have rightly drawn growing research attentio...
Vision-Language Models (VLMs) have advanced rapidly in multimodal perception and language understanding, yet it remains unclear whether they can reliably ground language into spati...
Large language and vision-language models increasingly power agents that act on a user's behalf through command-line interface (CLI) harnesses. However, most agent benchmarks still...
Synthetic benchmark for privacy-preserving and fairness-aware ranking under signal loss
TL;DR: I built a 3-LLM code reviewer (Claude + GPT-5 + Gemini that deliberate). My synthetic-bug...
Shipped this weekend: a 3-agent blind cross-lab evaluation workflow on heym, MIT licensed, callable...
Frontier LLMs corrupt 25% of documents in long workflows per new benchmark, while a Fields Medalist reports ChatGPT 5.5 Pro solving PhD-level math. Mayo Clinic AI detects pancreati...
We present Metal-Sci, a 10-task benchmark of scientific Apple Silicon Metal compute kernels spanning six optimization regimes (stencils, all-pairs in n-body problems, multi-field B...
📰 StepAudio 2.5 TTS Ranks #1 Chinese Voice Model on Artificial Analysis Leaderboard 2026StepAudio 2.5 TTS has emerged as China's leading voice model, ranking among the top three gl...
Show HN: Nexa-gauge – Cache/cost-aware graph-based eval for LLM and RAGNexa-gauge는 LLM, RAG, 에이전트 시스템의 생성 결과를 평가하기 위한 파이썬 패키지이자 CLI 도구로, 그래프 기반 평가 파이프라인을 통해 반복 가능한 품질 지표와 비용 추정, 캐시...
TL;DR We added architectural context to AI coding agents via MCP and tested on SWE-bench...
Following the recent achievement of gold-medal performance on the IMO by frontier LLMs, the community is searching for the next meaningful and challenging target for measuring LLM ...
While video foundation models excel at single-shot generation, real-world cinematic storytelling inherently relies on complex multi-shot sequencing. Further progress is constrained...
🧠 Researchers present IatroBench, a pre-registered benchmark that measures potential harms caused by AI safety interventions themselves. The study examines whether safety measures ...
We tested GPT-5.5 Instant for 3 straight days.Not benchmark theater.Real workflows: • SEO content• coding• research• automation• long promptsVerdict: GPT-5.5 Instant feels signific...
Lifecycle-Aware Memory for long-horizon LLM agents — 66.05% on PaperBench, 94.66% on SurveyBench, 10 peer-reviewed acceptances at FSE/ICML/TOSEM/AEI/ICoGB
Agent-skills-eval – Test whether Agent Skills improve outputshttps://github.com/darkrishabh/agent-skills-eval#HackerNews #AgentSkills #Eval #Testing #Skills #Outputs #AIResearch #M...
Show HN: Agent-skills-eval - Test whether Agent Skills improve outputshttps://github.com/darkrishabh/agent-skills-eval#HackerNews #Tech #AI
The benchmarks worth caring about measure something a customer would pay for. “Can this agent ship a...
Current LLM safety benchmarks are predominantly English-centric and often rely on translation, failing to capture country-specific harms. Moreover, they rarely evaluate a model's a...
Despite the growing popularity of Multimodal Domain Generalization (MMDG) for enhancing model robustness, it remains unclear whether reported performance gains reflect genuine algo...
Many deployments must compare candidate language models for safety before a labeled benchmark exists for the relevant language, sector, or regulatory regime. We formalize this sett...
Accurately understanding the intent behind speech, conversation, and writing is crucial to the development of helpful Large Language Model (LLM) assistants. This paper introduces I...