TEMPO extends a large language model's vocabulary with discrete codes from a context-aware transformer tokenizer using f...

TEMPO extends a large language model's vocabulary with discrete codes from a context-aware transformer tokenizer using finite scalar quantization, allowing signal patches to be mapped to a 625-code vocabulary and processed alongside text.Source: arXiv cs.CLhttps://arxiv.org/abs/2611.04482#MachineLearning