LocalLLaMA
Here is my llama.cpp NVFP4/MXFP6 GGUF quantizer tool
Hello everyone I wanted to share what I've been working on. I started writing NVFP4 kernels for llama.cpp last year and needed the ability to quantize NVFP4 GGUFs, so this project started as an NVFP4 quantizer. It's since become much larger. I would love to get more help to improve it. This is what I call the advanced-quantizer-tool (MIT license). This is used to create NVFP4 and MXFP6 models into GGUFs directly. But it can do much more. The latest model I've made with it are here: Qwen...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit