You Can't Regex Your Way Out of a Good Argument
·2042 words·10 mins
I spent the winter teaching people to defend against prompt injection. Layer your defenses, I said, and you can ship systems you trust. I still believe that. But I found an attack that walks through every layer I described, and it has no payload at all. It is just an argument. You push back on the model’s conclusion with confidence, and it folds. Every time. No guardrail fires because no guardrail was watching the conclusion itself. That changes things.