LocalLLaMA
Devs - do you use Mistral Medium 3.5 (128b dense) and if so - thoughts?
I've picked up a 3-bit quant of this one (Unsloth - Q3_KS) - the best fit for my config right now. Normally I shy away from 3-bit quants but as this is a giant dense model, I figured... why not. I've tested it a few hours in my latest project, and it found a few things that my daily driver had missed. It seems really good. Slower than an MoE, but not bad for me (8 tok/sec - with KV 80k - quant K = q8_0 quant v = q5_0). I was wondering what other people thought after actually using it with co...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit