
LocalLLaMA
cyankiwi AWQ 4-bit — 26.05 update, NVFP4 + FP8 Dynamic quantization and benchmarks across Qwen3.6 4-bit quants
We are happy to share cyankiwi AWQ update: better AWQ implementation, now with NVFP4 and FP8 Dynamic quantization support. We measured KL divergence against the BF16 baseline for 4-bit Qwen3.6 quants, on synthesized Qwen3.6 BF16 GPQA Diamond responses. cyankiwi AWQ release comes out lowest on both the 27B dense and the 35B-A3B MoE. Qwen3.6-27B (dense) Model Weight size KLD Lorbus/Qwen3.6-27B-int4-AutoRound 17.69 GiB 0.031682 Intel/Qwen3.6-27B-int4-AutoRound 17...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit