Tech
What I learned trying to benchmark local LLMs honestly
I had eight models sitting on my laptop and no idea which one I should actually be using.
I could get tokens per second out of llama-bench . That told me llama3.2 was fast. It told me nothing about whether llama3.2 was good — whether the quantization I'd pulled had quietly broken something, whether the 8B model I'd been avoiding because it felt sluggish was worth the wait.
So I built the thing that measures both. It's called homebench, and most of the month I spent on it went not in...
Read the full discussion on Dev.to
This article was aggregated from Dev.to. Click to join the conversation.
View on Dev.to