LocalLLaMA
Anyone seen benchmarks comparing Gemma 4 4-bit QAT vs. 8-bit standard quants?
I'm trying to find out if anyone has done any benchmarking comparing the Gemma 4 4-bit QAT models (via Unsloth) against standard 8-bit non-QAT quants. I know QAT is supposed to retain a ton of accuracy compared to the baseline BF16, but I'm curious how a 4-bit QAT model actually fares against a traditional 8-bit PTQ. I've read some mixed feedback across different threads, but I haven't been able to find hard numbers or a direct head to head comparison between the two. Has anyone run any eval...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit