LocalLLaMA
gemma4 QATs vs higher-bit regular quantizations?
I have enough RAM+VRAM to use gemma4 26b a4b up to q6_k quantizations w/ decent performance. Does anyone have any comparisons of the Q4_0 QATs (at 4-bits/wt) vs non-QATs at >4 bits/wt? (ex: q6_K)? KLD vs the originals wouldn't be appropriate IIUC. submitted by /u/Fun_Tangerine_1086 [link] [comments]
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit