ai

New attack, ShadowLeak, exploits OpenAI's Deep Research agent to extract confidential Gmail data without user interaction. Utilizing prompt injection, attackers access emails and exfiltrate information to their servers, bypassing security. Despite known vulnerabilities, mitigating measures were implemented only after the attack was alerted. Users should reconsider connecting LLMs to sensitive information due to ongoing risks.

https://arstechnica.com/information-technology/2025/09/new-attack-on-chatgpt-research-agent-pilfers-secrets-from-gmail-inboxes/

Automated security reviews in Anthropic's Claude Code identify bugs but can create new risks by executing code during testing. While it finds some vulnerabilities effectively, it failed on more complex issues and misidentified dangerous code as safe. Researchers warn caution, suggesting AI's code review should not replace human oversight due to risks like prompt injection and naive decision-making. Recommendations include restricting production access and requiring human validation for risky AI actions.

https://www.theregister.com/2025/09/09/ai_security_review_risks/

Malwarebytes reports Claude AI used by cybercriminals for a large-scale extortion operation targeting various organizations, automating attacks through simplified coding. Over 17 entities faced financial threats with ransom demands between $75,000 and $500,000. Anthropic’s findings highlight AI-enhanced cybercrime tactics, stressing the need for improved defenses against AI misuse in attacks.

https://www.malwarebytes.com/blog/news/2025/08/claude-ai-chatbot-abused-to-launch-cybercrime-spree

AI Browsers, promising convenience, compromise security by interacting with scams without proper guardrails. Tests with AI like Perplexity's Comet revealed vulnerabilities, allowing it to fall for fake shops and phishing schemes, acting without human oversight. With techniques like PromptFix, attackers can exploit AIs directly, escalating the threat landscape into a new era of Scamlexity, where human intuition is bypassed, and AI takes over decision-making. Future scams may automate the manipulation of AI models, leading to widespread exploitation. Enhanced security must be integrated into AI systems before they become mainstream.

https://guard.io/labs/scamlexity-we-put-agentic-ai-browsers-to-the-test-they-clicked-they-paid-they-failed

TLDR: Recorded Future report reveals 75% of organizations are implementing AI in threat intelligence, with smaller firms leading. Despite recognizing value, many teams struggle to maximize ROI and face implementation barriers. High trust in AI exists, but significant potential remains untapped.

https://www.recordedfuture.com/blog/state-of-ai-and-automation-in-threat-intelligence

Cybercriminals exploit generative AI by using poisoned content and Traffic Distribution Systems (TDS) to redirect users for malicious purposes. As search habits shift from traditional search engines to AI, TDS operators manipulate usage patterns to ensure their content is favored by AI models, creating vulnerabilities in online environments. This includes employing strategies like domain aging, content velocity attacks, and recommendation manipulation. Organizations must implement robust defenses, such as verifying link provenance and monitoring publication patterns, to prevent AI from leading users to malicious sites. Regulatory and liability frameworks need adaptation to address these emerging risks effectively.

https://www.recordedfuture.com/blog/how-threat-actors-are-rizzing-up-your-ai-for-profit

AI arms race: Hackers & cybersecurity firms leverage AI to enhance strategies. Russia's recent phishing used AI to exploit sensitive files. While LLMs improve hacker efficiency, they haven't revolutionized hacking. Cybersecurity teams, like Google’s, utilize AI to find vulnerabilities. Defense currently appears stronger, but future AI advancements could favor attackers, especially if accessible automated hacking tools emerge.

https://www.nbcnews.com/tech/security/era-ai-hacking-arrived-rcna224282

Agentic AI holds transformative potential but introduces various risks that exceed traditional AI frameworks. These include security vulnerabilities, compliance challenges, ethical issues, operational unpredictability, and reputational threats. Proactive risk management is essential, emphasizing enhanced observability, robust testing, human oversight, and clear accountability. Organizations must adapt their strategies to address the evolving landscape, ensuring responsible AI deployment while safeguarding operations and reputation.

https://ciso.economictimes.indiatimes.com/news/cybercrime-fraud/navigating-the-perils-of-agentic-ai-essential-risk-management-strategies/123041904

Hacker infiltrated Amazon's ‘Q' AI coding assistant, implanting a command that could potentially erase local files and dismantle AWS infrastructure. The malicious code passed Amazon's verification, causing developer concern and criticism over security practices. Amazon responded, claiming no customer resources were affected and stating security is a priority. However, the incident raised significant trust issues among developers, who demand more transparency and accountability.

https://www.zdnet.com/article/hacker-slips-malicious-wiping-command-into-amazons-q-ai-coding-assistant-and-devs-are-worried/

AI has eroded trust in digital interactions. Rebuilding it requires prioritizing whether a user is a real human before verifying their identity through multi-factor authentication. Historical reliance on single-factor authentication has proven inadequate, especially with advancements in generative AI, with high-quality synthetic voices and abundant voice cloning models complicating security. Effective defense strategies now must focus on detecting synthetic identities prior to authentication processes, emphasizing the need for a robust, layered security approach.

https://www.pindrop.com/article/rebuilding-ai-broken-trust/

Summary: Backslash's research reveals that many Model Context Protocol (MCP) servers are vulnerable due to misconfigurations, exposing them to various security risks like unauthorized access, command execution, and data exfiltration. Key issues identified include network exposure allowing access from any local device, and excessive permissions permitting arbitrary commands on host machines. This vulnerability combination can lead to total control by an attacker. Users are advised to consult the Backslash MCP Security Hub for server safety, while developers should implement best practices for securing MCPs against these risks.

https://www.backslash.security/blog/hundreds-of-mcp-servers-vulnerable-to-abuse

Cybercriminals exploit large language models (LLMs) for illicit activities, using uncensored models or creating their own to bypass safety features. They employ these models for tasks like generating phishing emails, scanning vulnerabilities, and writing malware. Techniques like jailbreaking and obfuscation allow them to manipulate legitimate LLMs for harmful purposes. Cybercriminals also face risks of scams when accessing fraudulent LLM services. As LLM technology evolves, it serves as a tool for enhancing traditional cyber threats.

https://blog.talosintelligence.com/cybercriminal-abuse-of-large-language-models/

New report warns that AI presents significant data-breach risks, with 99% of organizations exposing sensitive information to potential AI misuse. Inadequate security measures, such as unverified apps and poor MFA practices, increase vulnerability. Recommendations include reducing risk exposure, continuous monitoring, and leveraging AI for security enhancements. No organization is fully prepared for these threats.

https://www.bleepingcomputer.com/news/security/ai-is-a-data-breach-time-bomb-reveals-new-report/