Pruning a large number of attention heads before inference can markedly reduce verbatim memorisation in GPT-2-XL while s...

Pruning a large number of attention heads before inference can markedly reduce verbatim memorisation in GPT-2-XL while slightly improving its language-modeling performance.Source: arXiv cs.CLhttps://arxiv.org/abs/2610.40974#MachineLearning

Read Original

Related