
LocalLLaMA
Gemma 4 E4B IQ2_XXS: + 140.54% Reasoning Performance From Tensor Level Quantization Allocation
iq2_xxs tensor level allocation recovered reasoning from 28.9 -> 69.5 at the same 3.3gb budget. https://huggingface.co/ByteOtter/gemma-4-E4B-it-CADA-IQ2_XXS I posted my Gemma 4 12B q3 result a couple days ago, where tensor level allocation gave me an +8.55% relative improvement over the category imatrix baseline. I pushed further down to iq2_xxs territory where the damage is more severe and the result is far more dramatic. Same idea as before, build imatrix from a category oriente...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit