Tech
How I benchmark an agent guardrail without lying to myself
It would be easy to write a benchmark for Doberman that makes Doberman look good: pick the attacks it already stops, skip the ones it doesn't, round up. I've tried to build the opposite of that - a methodology that reports the numbers whether they're flattering or not, and a results page ( docs/BENCHMARKS.md ) that lists what it missed before what it caught.
The metrics, and why there are three of them
Doberman is scored as a decision function over tool-call cases, not by driving...
Read the full discussion on Dev.to
This article was aggregated from Dev.to. Click to join the conversation.
View on Dev.to