Link to the article: KV Cache Quantization Benchmarks: KVarN, Precision Tail KLD benchmarks with BeeLlama.cpp v0.4.0 , fork of llama.cpp with more KV cache quantization options. Models: Qwen 3.6 27B Q5_K_S 64k context, Gemma 4 31B Q5_K_S 16k context Standard quants, extended: q6_0 and q6_1, and low-bit types from q2_0 to q3_1 KVarN: Variance-Normalized KV-Cache by Huawei, implemented in BeeLlama Precision Tail: keeping latest X tokens of KV cache in (B)F16, implemented in BeeLlam...