Tech
My Release Gate Passed. The Model It Shipped Answered 'Neutral' To Everything.
I trained a model that scored 4 percentage points better than the one I ship. It's gone. I
deleted it from the roadmap yesterday.
That's not the interesting part. The interesting part is how I found out it was broken — because
it passed every check I had, and then shipped a file that answered neutral to every question you
could ask it.
The setup
I maintain semantix , a local NLI judge for
compliance validation. The shipped model, nli-popia-v2 , is an 82M-param cross...
Read the full discussion on Dev.to
This article was aggregated from Dev.to. Click to join the conversation.
View on Dev.to