Tsinghua researchers open-sourced AKV, an SGLang-based serving system designed for LLM agent cache engineering. By allow...
Tsinghua researchers open-sourced AKV, an SGLang-based serving system designed for LLM agent cache engineering. By allowing agents to drop and reposition KV cache directly instead ...