A couple of day ago I posted about OmnisBench, our open benchmark for LLM routing, specifically our LLM Router OmnisRouter , and made a fuss about how you can re-grade every number yourself because we publish the actual model responses. Two commenters, deanlee and jugeni, very politely pointed out that the whole thing might be resting on a fib. Their point: HumanEval and GSM8K are old. The models have almost certainly read the answers. So when I cheerfully reported that the cheapest model a...