LocalLLaMA
NVFP4 on llama.cpp?
Hey everyone, Even through I check the subreddit daily, some things are a bit hard to grasp for me due to the speed at progress is made (really impressive!). I tried doing research using deepseek v4 but it left me even more puzzled. Recently I saw NVFP4 support being merged into llama.cpp. Since I have dual RTX 5060 Ti's, I would love to make use of it but I didn't fully grasp how. I also saw someone releasing NVFP4 quants of Gemma4 QAT, seen here: https://huggingface.co/melcheikh/gemma-...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit