Tech
Telling a cheap model to "think a little" made it slower and sometimes silent
I told a model to think less, and it started thinking.
That's the whole bug. But how I got there is worth writing down, because anyone putting one endpoint in front of several model families will probably hit some version of it.
The setup: I build a desktop app that can talk to a model three ways. A local model through Ollama, the user's own API key, or cloud credits that go through a small gateway I run (a Cloudflare Worker) in front of OpenRouter. In that last mode the app doesn't know w...
Read the full discussion on Dev.to
This article was aggregated from Dev.to. Click to join the conversation.
View on Dev.to