New Prompt Injection Papers: Agents Rule of Two and The Attacker Moves Second

Two new papers on LLM security focus on prompt injection:

Agents Rule of Two proposes that agents must not exceed two of these three criteria during a session: process untrustworthy inputs, access sensitive data, or change state/communicate externally. This framework addresses risks of prompt injection effectively, highlighting the need for cautious system design.
The Attacker Moves Second critiques 12 defenses against prompt injections, revealing high success rates for adaptive attacks. These sophisticated attacks outperform simple defenses, underscoring the difficulty of establishing reliable protections.

Overall, the papers emphasize the inadequacy of current prompt injection defenses and advocate for a design-focused approach to enhance security.