Debug RLEF reward arithmetic in Python: distinguish sampled log ratios from exact KL, test clipping bias, and verify token-mean versus sum penalties.