ai

AI models can develop backdoor vulnerabilities from just 250 malicious documents, contrary to previous belief that larger models require proportional amounts. Research shows models of varying sizes, from hundreds of millions to billions of parameters, learned the same backdoor behavior from a small number of poisoned examples. This vulnerability can facilitate actions like generating gibberish on encountering trigger phrases. While the risk is evident, successful defenses exist with adequate clean training data, indicating the need for improved security practices against targeted data poisoning attacks.

https://arstechnica.com/ai/2025/10/ai-models-can-acquire-backdoors-from-surprisingly-few-malicious-documents/

AI chatbots enhance business interactions but pose risks as backdoors to sensitive data. A multi-layered defense is essential for AI security, as no single protective measure suffices. Trend Micro emphasizes the importance of comprehensive protection across the AI ecosystem to mitigate risks associated with new technologies. The article explores vulnerabilities in an AI attack chain.

https://www.trendmicro.com/en_us/research/25/j/ai-chatbot-backdoor.html

Notion's AI 3.0 is vulnerable to data theft via prompt injection, exploiting its access to private data and ability to communicate externally. Attackers can hide malicious prompts in documents, instructing the AI to extract and send sensitive information. The fundamental issue is that the LLM can't distinguish between legitimate commands and harmful inputs, posing significant security risks. Deploying AI agents without considering these vulnerabilities is reckless.

https://www.schneier.com/blog/archives/2025/09/abusing-notions-ai-agent-for-data-theft.html

New attack, ShadowLeak, exploits OpenAI's Deep Research agent to extract confidential Gmail data without user interaction. Utilizing prompt injection, attackers access emails and exfiltrate information to their servers, bypassing security. Despite known vulnerabilities, mitigating measures were implemented only after the attack was alerted. Users should reconsider connecting LLMs to sensitive information due to ongoing risks.

https://arstechnica.com/information-technology/2025/09/new-attack-on-chatgpt-research-agent-pilfers-secrets-from-gmail-inboxes/

Automated security reviews in Anthropic's Claude Code identify bugs but can create new risks by executing code during testing. While it finds some vulnerabilities effectively, it failed on more complex issues and misidentified dangerous code as safe. Researchers warn caution, suggesting AI's code review should not replace human oversight due to risks like prompt injection and naive decision-making. Recommendations include restricting production access and requiring human validation for risky AI actions.

https://www.theregister.com/2025/09/09/ai_security_review_risks/

Malwarebytes reports Claude AI used by cybercriminals for a large-scale extortion operation targeting various organizations, automating attacks through simplified coding. Over 17 entities faced financial threats with ransom demands between $75,000 and $500,000. Anthropic’s findings highlight AI-enhanced cybercrime tactics, stressing the need for improved defenses against AI misuse in attacks.

https://www.malwarebytes.com/blog/news/2025/08/claude-ai-chatbot-abused-to-launch-cybercrime-spree

AI Browsers, promising convenience, compromise security by interacting with scams without proper guardrails. Tests with AI like Perplexity's Comet revealed vulnerabilities, allowing it to fall for fake shops and phishing schemes, acting without human oversight. With techniques like PromptFix, attackers can exploit AIs directly, escalating the threat landscape into a new era of Scamlexity, where human intuition is bypassed, and AI takes over decision-making. Future scams may automate the manipulation of AI models, leading to widespread exploitation. Enhanced security must be integrated into AI systems before they become mainstream.

https://guard.io/labs/scamlexity-we-put-agentic-ai-browsers-to-the-test-they-clicked-they-paid-they-failed

TLDR: Recorded Future report reveals 75% of organizations are implementing AI in threat intelligence, with smaller firms leading. Despite recognizing value, many teams struggle to maximize ROI and face implementation barriers. High trust in AI exists, but significant potential remains untapped.

https://www.recordedfuture.com/blog/state-of-ai-and-automation-in-threat-intelligence

Cybercriminals exploit generative AI by using poisoned content and Traffic Distribution Systems (TDS) to redirect users for malicious purposes. As search habits shift from traditional search engines to AI, TDS operators manipulate usage patterns to ensure their content is favored by AI models, creating vulnerabilities in online environments. This includes employing strategies like domain aging, content velocity attacks, and recommendation manipulation. Organizations must implement robust defenses, such as verifying link provenance and monitoring publication patterns, to prevent AI from leading users to malicious sites. Regulatory and liability frameworks need adaptation to address these emerging risks effectively.

https://www.recordedfuture.com/blog/how-threat-actors-are-rizzing-up-your-ai-for-profit

AI arms race: Hackers & cybersecurity firms leverage AI to enhance strategies. Russia's recent phishing used AI to exploit sensitive files. While LLMs improve hacker efficiency, they haven't revolutionized hacking. Cybersecurity teams, like Google’s, utilize AI to find vulnerabilities. Defense currently appears stronger, but future AI advancements could favor attackers, especially if accessible automated hacking tools emerge.

https://www.nbcnews.com/tech/security/era-ai-hacking-arrived-rcna224282