LocalLLaMA
DeepSeek V4 Flash 0731 - Happy Numbers (700pp/18tg) and Thoughts
Originally, I was only getting around 140pp/s and about 21tg/s, but the config with -b 8192 -ub 8192 --cpu-moe is vastly superior, let's say 700pp/s and 18tg/s in the most relevant range. Test System : CPU: Threadripper 5965WX (24c/48t) RAM: 512GB ( 8 x64 DDR4 ECC REG 2400 ) GPU: RTX 5090 (PCIe 4 x16, 450W) Command ./build/bin/llama-server \ -hf unsloth/DeepSeek-V4-Flash-0731-GGUF:UD-Q8_K_XL \ --temp 1.0 --top-p 1.0 --min-p 0.0 --host 0.0.0.0 \ --ctx-size 262144 --no-...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit