The first number I track when evaluating a free model route is not the public benchmark delta. It is the semantic failure count in a 100-case golden set. A public benchmark can be flat while your application's specific tool calls, JSON schemas, and failure states move. I start with a golden set: 100–300 request/response pairs captured from the current route. Each case stores the expected tool call and a small list of acceptable result patterns. The set must include ambiguous prompts, permissi...