LocalLLaMA
Qwen3.6-MTP-27B on Tesla V100 @ 55 TPS (llama.cpp) — Any way to push this higher without quality loss?
Hey everyone, I'm running Qwen3.6-MTP-27B-MTP (Q4_K_M) with llama.cpp server on a Tesla V100 , and I'm currently getting around 55 tokens/sec . I'm trying to find out whether there are any configuration changes that could increase throughput further without reducing output quality . 55 TPS seems lower than I expected for MTP on a V100, but I may be missing something obvious. Current command: llama-server \ -m ../NewModels/Qwen3.6-MTP-27B-Q4_K_M.gguf \ --port 9932 \ --host 0.0...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit