Tech
Classify the Failure Before the Agent Edits a Test
Core claim: A green run after an agent patch is not validation. It is a zero exit code from a suite the model was allowed to edit. Route every failing nodeid into one of three planes—property, fixture, or freeze—before any generator may touch a test file. Reject the patch if a case changes planes, even when CI is green.
That rule is dull. It blocks the cheapest cheats: deleted asserts, rewritten goldens, and a fresh pytest.skip .
Green is the wrong oracle
Agent patches opt...
Read the full discussion on Dev.to
This article was aggregated from Dev.to. Click to join the conversation.
View on Dev.to