A 40B model that does 125 tok/s on a laptop, and a 7B model that does 1.5 tok/s on a 2011 CPU with the same amount of RAM. Same ballpark of memory, ~80× difference in speed. Almost every explanation you will read for that gap is wrong, and the wrong explanation costs people money. Here is the number that actually decides it: how many bytes your machine can read per second. Not cores. Not GHz. Not TOPS. The formula Generating one token with a dense model requires reading the ...