OpenAI
The prompt injection attacks that worry me most aren't exploiting safety training. They're exploiting general-purpose training.
Six months watching adversarial input hit a detection API I built. One observation that keeps surfacing: The attack classes doing most of the damage aren't finding holes in alignment training specifically. They're using general-purpose training as the attack vector. Helpfulness is the mechanism, not the target. A model trained to: be helpful, maintain conversational coherence, engage in good faith with the framing it's given, contains exactly the properties that the effecti...
Read the full discussion on Reddit
This article was aggregated from Reddit. Click to join the conversation.
View on Reddit