LocalLLaMA
Try out this "high" reasoning mode for 27B (tested on VLLM)
After a lot of tweaking, I have come to the conclusion that 27B lacks a reasoning mode that is between low and xhigh. The "medium" mode isn't actually medium, it erases the explicit instructions to the model. When medium is enabled, the model acts very differently - to me it looks like it regresses to behaving more like 3.6, and loses some of the 3.8 gains. Low and high mode behavior in the model seem to be triggered almost exclusively by using certain keywords in the reasoning ins...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit