LocalLLaMA
I just realized how good MoE models are for consumer hardware
I've been tinkering around with LLM for a while now, started with LM Studio like probably all of us and wanted to go into headless selhosted model so that I can use my macbook and still use my AI models. I've been using Qwen 3.6 (and 3.5) 27B on my main computer which has a Ryzen 7 3800X, a 7900XT, 32Gb of RAM and that thing was pretty sloooooow even with MTP enabled. You can probably call this a skill issue as I'm not familiar with llama.cpp forest of arguments yet despite reading the docum...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit