LLMs and coding agents pose significant security risks, increasing vulnerabilities in systems. The unpredictability of LLMs leads to issues like prompt injection, where attackers exploit them to take unauthorized actions. New agent technologies further amplify risks by automating actions without adequate user oversight. Vulnerabilities can be hidden in code or instructions, leading to severe consequences like remote code execution (RCE) attacks. While suggestions exist to mitigate risks, the temptation to use these tools may compromise security, warning against treating LLMs as reliable.
https://garymarcus.substack.com/p/llms-coding-agents-security-nightmare