Collaborative Personalized Preference Alignment for LLMs under Data Deficiency
Real-world users often exhibit highly heterogeneous preferences over multiple objectives for LLM responses. A lightweight aligner can tailor these responses to individual preferenc...