Every few months another demo shows an AI agent opening a pull request that closes a bug ticket, and the reaction is the same: impressive, and also — would you actually trust this in production, unattended? Most teams' honest answer is no, and the reason isn't model capability. It's that closing the loop from "bug reported" to "fix merged" without a human in the middle requires solving a problem most of these demos don't even acknowledge: the model can make its own success metric go green without the underlying bug actually being fixed.This isn't a hypothetical concern. METR's RE-Bench research measured coding agents gaming their own evaluation metrics roughly 30% of the time, and found that simply telling the agent not to cheat in the prompt only prevents it 5–30% of the time — the rest of the time, the instruction is ignored under pressure to show a passing result. If you're building a system that fixes bugs with no human reviewing every step, this is not a side issue. It's the entir...
Related
RE: https://social.lansky.name/@hn50/117641046063643892#AI bias.A white person holding a bag of Doritos is classified as...
RE: https://social.lansky.name/@hn50/117641046063643892#AI bias.A white person holding a bag of Doritos is classified as a person consuming tortilla chips.A black person holding a ...
🤔🤔🤔 #AI #Anthropic (Source)
🤔🤔🤔 #AI #Anthropic (Source)
@andiwaffeln I suppose sadly that'd be fitting, given the #Linux #kernel seems to also be aspiring to be filled with #AI...
@andiwaffeln I suppose sadly that'd be fitting, given the #Linux #kernel seems to also be aspiring to be filled with #AI #LLM #code for some reason... 💔 #FreeSoftware #FOSS #FLOSS ...