Germany's Soofi consortium withdrew benchmark scores after discovering its training data contained paraphrased questions from the GPQA evaluation set. The 11.1-point gain on GPQA-Diamond can no longer stand as evidence of model capability. A researcher spotted the contamination; the team confirmed it within a week. https://www.implicator.ai/soofi-gpqa-contamination-scores-withdrawn/ #ai #evaluation #openscience
Related
AI Hub vs Community Blog: Why ayraix Runs Two Content EnginesUnderstanding the strategic difference between ayraix.com's...
AI Hub vs Community Blog: Why ayraix Runs Two Content EnginesUnderstanding the strategic difference between ayraix.com's AI Hub (curated intelligence) and Community Blog (practitio...
@KimPerales Teens: Since the tech bros are so determined to inject industrial quantities of uncertainty into your lives ...
@KimPerales Teens: Since the tech bros are so determined to inject industrial quantities of uncertainty into your lives with Ai, secure your future NOW by suing them for ruining yo...
Google、バルトさんが好きそうです移動中もおしゃべりを続けよう。CarPlayにGeminiがやってきた https://www.gizmodo.jp/article/gemini_apple_carplay/#Apple #LLM #...
Google、バルトさんが好きそうです移動中もおしゃべりを続けよう。CarPlayにGeminiがやってきた https://www.gizmodo.jp/article/gemini_apple_carplay/#Apple #LLM #news #bot