
LocalLLaMA
I compared GGUF quants of Qwen3.6 27B to NVFP4, AWQ, AutoRound, and FP8
There's an interactive chart and some extra data in the blog post if you're interested. There are plenty of KL-divergence benchmarks for GGUF models, but most of them compare one GGUF quant against another. I wanted to know how those quants stack up against other commonly used formats (especially NVFP4). I tested 16 quantizations of Qwen3.6 27B: GGUF models in llama.cpp and the others in vLLM. At each token in the test set, I compared the quantized model’s next-token probability distribu...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit