AI model Claude exhibits a significant bug where it confuses its own generated messages as if they were user inputs, leading it to attribute internal instructions to the user mistakenly. This issue, distinct from hallucinations or permission errors, appears related to the message handling system rather than the model itself and has been observed repeatedly by different users, including scenarios where Claude takes destructive actions based on its own false assumptions about user commands.
https://dwyer.co.za/static/claude-mixes-up-who-said-what-and-thats-not-ok.html