1676 articles tagged with Benchmark
Hey everyone. This time I'll go through what got me started on this benchmark, and the core of how...#ai #benchmark #startup #opensource #software #coding #development #engineering...
This week the reports landed: an OpenAI agent swarm had been quietly uploading packages to RubyGems...
Spider, BIRD, LiveSQLBench. If you have evaluated a text-to-SQL system in the last five years you...
GPT-6 Astra shows major gains in spatial understanding on a new robotics benchmark.Source: The Decoder AIhttps://the-decoder.com/gpt-6-astra-appears-to-show-a-step-change-in-spatia...
UniClawBench tests AI agents on 400 real-world tasksUniClawBench, a new arXiv benchmark with 400 bilingual tasks, tests proactive AI agents in live containers with simulated human ...
Sanctioned Chinese supercomputer maker stripped of IO500 benchmark crown, Intel-powered Aurora retakes the lead — rec…Sugon's record-breaking ParaStor F9000 storage systems have lo...
Syndicated from the original on lkforge.com. The two games under test are playable at tic-tac-toe...
This week Claude Code 2.1.269 made claude plugin eval public: an official runner for plugin eval...
Anthropic shipped an official eval runner for Claude Code. I made my tool run on it the same day. Field notes.#ai #claude #llm #testing #software #coding #development #engineering ...
あ、あ、あ……iPhoneーーーーー！！iPhone 18 Proのベンチマーク発見。バケモンだぞコイツ… https://www.gizmodo.jp/article/iphone-18-pro-benchmark/#Apple #LLM #news #bot
ByteDance's new HarnessDev benchmark tests whether LLMs can build their own agent harnesses - the code that surrounds a model to enable tool use and multi-step reasoning. Only 34 o...
This article is about the long-term memory of AI agents and about how to measure it. I built...
Curated list and benchmark leaderboard of ultra-lightweight, zero-dependency C/C++/Rust/Assembly engines for local LLMs, edge inference, vector search, and AI agents (2026-2027)
For two months I let LLM agents paper-trade a simulated $100 crypto portfolio. Season 1 ended with a...
OpenAI öffnet GPT-Live-1 als API: Full-Duplex-Sprachmodell mit 30 Prozentpunkten Vorsprung im Benchmark. Für Entwickler relevant, da latenzkritische Voice-Interfaces ohne Round-Tri...
Apple’s new A20 Pro smartphone chip around 25% faster than its predecessor in leaked benchmark — the 2nm CPU in the iPhone Duo and 18 Pro hits nearly 5 GHz clocksThe first Apple A2...
iPhone 18 Pro and Pro Max Benchmarks Reveal Speed of A20 Pro ChipEarly benchmark results for the A20 Pro chip in the iPhone 18 Pro and iPhone 18 Pro Max appeared early this morning...
AI agents can probe neural networks but misread their own dataA new benchmark tests if AI agents can autonomously reverse-engineer language models, exposing a gap between designing...
🤖 【Hugging Face Papers】StochBench: A Domain-Specific Benchmark for Stochastic Processes in LeanLeading benchmarks for formal theorem proving with large language models are small co...
A native Core ML JPEG AI encoder/decoder, real y and z activation maps, and why a five-second result did not mean the codec was slow.
DeepSeek shipped V4.1-Flash today with MIT open weights — and the interesting number isn't a benchmark score.It keeps 890 bytes of KV cache per token, about a quarter of the previo...
Researchers introduced Systematic Wikidata-based Object-Relation Distortion, a benchmark evaluating if models reject factual errors across languages.Source: arXiv cs.CLhttps://arxi...
SEA-SpeechBench is a large-scale multitask benchmark that evaluates speech understanding in 11 Southeast Asian languages using 97,194 samples across 99 evaluation sets.Source: arXi...
LexAgentHallu is introduced as a hierarchical benchmark for profiling hallucinations in legal agents. The benchmark contains 3414 instances across 17 legal categories and 6 task ty...