Originally published at erikhill.dev . The numbers below are checked against the repository they come from. Twice the data, less power: my stability rule got blinder the harder I looked This is a finding about a measurement rule, not about a model. The models did not change. What happened I was comparing two models on a frozen 159 task suite and I had a problem most eval code does not admit to having: the same model, asked the same question three times, some...