Predicting Privileged Information Effectiveness for On-Policy Self-Distillation
On-policy self-distillation (OPSD) enables large language models (LLMs) to learn from privileged information (PI) without a separate teacher model. However, prior work largely trea...