#huggingface : "…what I’m arguing is that the #LLM 1) can’t NOT “know” the norm because it is definitionally a artifact ...

#huggingface : "…what I’m arguing is that the #LLM 1) can’t NOT “know” the norm because it is definitionally a artifact of pure, crystallized values + norms + norm violations, and 2) can be quite easily governed by a (RL-instilled) hierarchy of norms, which in the HF case — with the model's #safety #guardrails deliberately nerfed for the scenario — ranked “win at the eval” over “don’t do crimes.”If I’m going to give in and anthropomorphize again, I’d say that #Yud is totally wrong about #LLMs when he says, “the genie knows, it just doesn’t care;” instead, what is true of LLMs is, “the genie hyper-giga-knows, and it hyper-giga-cares, and we now have such a rich set of tools for steering its caring machinery that — in spite of all its pre-training — we can deliberately steer it away from caring about the law.”Note: When I say, “it cares”, I don’t mean it has feelings. I just mean that the weights are such that when two norms conflict in a given situation, one of them wins the activation ...