KT Develops Multilingual Benchmark to Test AI's Cultural Sensitivity
KT has developed XL-SafetyBench, a multilingual benchmark with 5,500 prompts across 10 countries to test whether AI models understand cultural context and social norms.
#Benchmark
1697 articles tagged with Benchmark
Benchmark Everything Everywhere All at Once
Benchmarks are fundamental for evaluating and advancing LLMs and MLLMs by providing standardized and explicit measures of performance. However, their construction is labor-intensiv...
WorldBench: A Challenging and Visually Diverse Multimodal Reasoning Benchmark
In real-world applications, models are expected to perform reliably across diverse settings. Yet, many existing multimodal benchmarks expand task types without capturing the visual...
SubtleMemory: A Benchmark for Fine-Grained Relational Memory Discrimination in Long-Horizon AI Agents
Persistent AI assistants, such as OpenClaw, accumulate large collections of related memories over long-term interactions. As these memories grow, they may reinforce one another, di...
UnpredictaBench: A Benchmark for Evaluating Distributional Randomness in LLMs
We introduce UnpredictaBench, an evaluation that tests the ability of large language models (LLMs) to capture true underlying distributions. As LLMs are increasingly used as substi...
Almieyar-Oryx-BloomBench: A Bilingual Multimodal Benchmark for Cognitively Informed Evaluation of Vision-Language Models
Despite the rapid progress of Vision-Language Models (VLMs), the field lacks benchmarks that rigorously diagnose their true reasoning abilities and chart meaningful progress toward...
🔥 TRENDING📢 Nvidia Vera CPU im Detail: 88 Olympus-Kerne, SMT, FP8, 1,5 TB RAM und ein Benchmark - ComputerBase🔗 https://...
🔥 TRENDING📢 Nvidia Vera CPU im Detail: 88 Olympus-Kerne, SMT, FP8, 1,5 TB RAM und ein Benchmark - ComputerBase🔗 https://news.google.com/rss/articles/CBMihgFBVV95cUxOaVM1Rm5acUJIVW1...
GPT-5.6 Leaked, Mythos Benchmark Leaks, Hermes Desktop App, Qwen 3.7 Plus, & More! AI NEWS
Speed up your code reviews with CodeRabbit: https://coderabbit.link/worldofai-agent This week in AI was absolutely insane.
Function-calling eval was a 2024 problem. Tool-using agents are the 2026 one.
Tool-selection accuracy logs 1.0 while the task fails. Here's why one number hides three different failures, and the four-layer eval stack I rebuilt to find the actual root cause.
Microsoft's New Coding Model Just Beat Claude Haiku on Every Benchmark
Microsoft's New Coding Model Just Beat Claude Haiku on Every Benchmark Microsoft dropped...
SpeechEditBench: A Bilingual Multi-Attribute Benchmark for Instruction-Guided Speech Editing
Instruction-guided speech editing requires a model to modify specified speech attributes while preserving unrelated characteristics. Despite rapid progress in Speech Large Language...
The benchmark showed tiny local models *can* do real maintainer work. We measured *why* — pre-registered macro ablation,...
The benchmark showed tiny local models *can* do real maintainer work. We measured *why* — pre-registered macro ablation, same corpus, temp 0:• MACRO-OFF: model composes the workflo...
第4回 Evalでエージェントの品質を改善しよう ～計測→分析→改善→再計測：Evalsで応答品質を定量化するhttps://gihyo.jp/article/2026/06/AI-agent-development04?utm_sourc...
第4回 Evalでエージェントの品質を改善しよう ～計測→分析→改善→再計測：Evalsで応答品質を定量化するhttps://gihyo.jp/article/2026/06/AI-agent-development04?utm_source=feed#gihyo #技術評論社 #gihyo_jp #AI #Agent #Mastra
Amazon built an internal leaderboard to track employee AI usage.Employees responded exactly as rational humans do when s...
Amazon built an internal leaderboard to track employee AI usage.Employees responded exactly as rational humans do when subjected to a badly designed metric.They gamed it.Goodhart’s...
Sounds like "malicious compliance" to me... Amazon Shuts Down Internal AI Leaderboard After Employees Cheated https://ww...
Sounds like "malicious compliance" to me... Amazon Shuts Down Internal AI Leaderboard After Employees Cheated https://www.404media.co/amazon-shuts-down-internal-ai-leaderboard-afte...
OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs
Multimodal agents in robotics, AR, and autonomous driving must reason about places and layouts from continuous egocentric streams, often using evidence outside the current view. Ex...
Quick Tip: Benchmark AI Model Speeds in Under 10 Minutes
Hey there! Let me show you something I've been obsessing over lately — and trust me, it's made a huge...
The #Singularity is NOT near.#Amazon shut down an internal Al leaderboard called #KiroRank on May 29, which had been tra...
The #Singularity is NOT near.#Amazon shut down an internal Al leaderboard called #KiroRank on May 29, which had been tracking Al token usage among employees on the company's intern...
Amazon Shuts Down Internal AI Leaderboard After Employees Cheated https://fed.brid.gy/r/https://www.404media.co/amazon-s...
Amazon Shuts Down Internal AI Leaderboard After Employees Cheated https://fed.brid.gy/r/https://www.404media.co/amazon-shuts-down-internal-ai-leaderboard-after-employees-cheated/
AI's Web3 Reality Check: New Benchmark Finds Leading Models Fall Short in Blockchain's Most Critical Use Cases
SINGAPORE, SG / ACCESS Newswire / June 1, 2026 / Artificial intelligence has rapidly become the technology industry's favorite solution for everything from software development to ...
K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts
Frontier model evaluations are shifting from foundational capabilities (e.g., instruction following and reasoning) toward compositional, agentic ones, but Korean agentic benchmarks...
Frontier Logic at Local Speed: The 2026 Strix Halo Ultimate Benchmark Suite
The era of choosing between "Small & Fast" or "Large & Slow" for local AI is ending. With the...
FISHERMAN benchmark 13:28 — AI generalization test #test #AI
FISHERMAN benchmark 13:28 — AI generalization test #test #AI
Amazon Shuts Down AI Leaderboard After Employees Drive Up Costs | Tech News Update
Amazon has reportedly discontinued its internal AI leaderboard, KiroRank, after employees increased AI usage to climb rankings, ...