Tech
[Local LLM] 125B 模型 Q4 量化， xHigh256k 调优峰值 64tps。附部署和调优源码
Strata windows
96G 显存分配 32G RAM
修了识图，去拒答
配置：q4xs 量化、上下文 256K ，配置如下，96G 显存分配 32GRAM
峰值 64.63TPS
智商类似不降智 opus4.6
https://github.com/xszwow/Strata
Read the full discussion on V2EX
This article was aggregated from V2EX. Click to join the conversation.
View on V2EX