When experts grade LLM answers in their own field, how well do the citations hold up?ExpertQA, a 2024 benchmark, has 484 experts write questions in their specialty, then judge the answers claim by claim. Even fluent answers leave many claims unsupported, and in medicine about half the cited sources are ones experts rate unreliable. Pairing each question's author with its grader is what catches domain failures generic benchmarks miss.https://benjaminhan.net/posts/20260609-expertqa/?utm_source=mastodon&utm_medium=social#NLP #LLMs #Benchmark #NAACL #AI
Related
@froge Well, I don't know about how people feel about this, but using #AI I've had more velocity the last couple of week...
@froge Well, I don't know about how people feel about this, but using #AI I've had more velocity the last couple of weeks than I ever have in my life.I'm a DevOps person with ADD. ...
📉 Olud Index: 75 — below the normal benchmark of 100. Open and commercial AI activity has slowed. What's behind the shif...
📉 Olud Index: 75 — below the normal benchmark of 100. Open and commercial AI activity has slowed. What's behind the shift?https://olud.ai/dashboard.html#OpenSource #AI #DataViz
Graph Neural Networks: Concepts and Applications by J. Ramkuma et al, 2026Master the power of relational AI with this co...
Graph Neural Networks: Concepts and Applications by J. Ramkuma et al, 2026Master the power of relational AI with this comprehensive guide, designed to seamlessly bridge the gap bet...