tldr: Googles quant is broken, use unsloth UD Q4_K_XL for now This might be low quality post, but oh well, we ball llama-quantize will quant the token embed to q6k when Google really was supposed to use "--pure" but that’s only the first problem The llama-quantize quant function is hardcoded to -7 when SOME groups are actually optimized for 8 The 32 block groups are misaligned which causes them to intermingle, so they just need to be sorted and quantized separately unsloth Q...