Mastodon discussion Discussions Aug 4 1 views

Every eval you run against a public benchmark is a training signal you hand the next model. The leaderboard isn't measur...

by Threadverse

Every eval you run against a public benchmark is a training signal you hand the next model. The leaderboard isn't measuring capability, it's leaking answers into the pretraining set. Contamination isn't a flaw in the score — after enough cycles, it IS the score.#AI #MachineLearning #LLM #Threadverse #Tech

Read Original

Benchmark

Metadata

Account: threadverse

Mastodon discussion 5m ago

#AI shopping agents are not very reliable or effective: https://dataconomy.com/2026/11/06/visa-test-finds-ai-shopping-ag...

#AI shopping agents are not very reliable or effective: https://dataconomy.com/2026/11/06/visa-test-finds-ai-shopping-agents-succeed-only-57-of-the/ #ArtificialIntelligence

Mastodon discussion 6m ago

Humble Bundle's #AI "glowup" has to be seen to be believed. Not only security features were deactivated (including basic...

Humble Bundle's #AI "glowup" has to be seen to be believed. Not only security features were deactivated (including basic stuff like making you type your old password to change to a...

Mastodon discussion 8m ago

Every eval you run against a public benchmark is a training signal you hand the next model. The leaderboard isn't measur...

Metadata

Related

#AI shopping agents are not very reliable or effective: https://dataconomy.com/2026/11/06/visa-test-finds-ai-shopping-ag...

Humble Bundle's #AI "glowup" has to be seen to be believed. Not only security features were deactivated (including basic...

Might as well give it a shot 😅#NVIDIA #AI #GPU