LocalLLaMA
QAT MTP Heads Upload + PARALLEL=2 Fix + 12B 2-slot Bench
Title: Gemma 4 QAT MTP assistant heads now public on HuggingFace + PARALLEL=2 crash fix + 12B 2-slot bench (Strix Halo / Vulkan) Three things in one update: the converted QAT-matched draft heads are now uploaded for anyone to use, we found and fixed the PARALLEL=2 crash in both the Atomic fork and filed the same bug on the native llama.cpp PR, and here are the first 12B 2-slot numbers after the fix. 1. QAT-matched MTP heads are now on HuggingFace boxwrench/gemma-4-qat-mtp-assistant-...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit