Saw this post here yesterday: KVarN: new KV-cache quant from Huawei. 3–5× KV cache compression with actual speed-up instead of slow-down, and unlike TurboQuant it holds up on reasoning (Apache 2.0, vLLM single flag) Cheap KV cache with good precision? Sign me up! Oh, vLLM only... Wait, I do have my own llama.cpp fork , and I do have an extensive reference for KLD benchmarking . I should act! And so I acted. Until 6 am. So now KVarN is implemented in a publicly available BeeLlama....