Tracking Threats, Incidents, and Vulnerabilities
Tracking Threats, Incidents, and Vulnerabilities
AI enhances DDoS attacks by enabling easy access to sophisticated tools, accelerating launch times, adapting strategies in real-time, and bypassing defenses through human-like bot behavior. Businesses need to proactively use AI for robust defense strategies against these evolving threats.
https://www.techradar.com/pro/5-ways-ai-is-supercharging-ddos-attacks
Prompt injection is a security challenge in AI, where attackers manipulate AI responses using malicious instructions in user inputs. As AI gains more capabilities and access to sensitive data, protecting users from these risks is crucial. OpenAI employs a multi-layered defense approach, including safety training, monitoring, security protections, user controls, red-teaming, and a bug bounty program. Users are advised to limit data access, verify agent actions, provide explicit instructions, and stay informed about security. Ongoing research aims to enhance AI robustness against these attacks and ensure safe interactions.
TLDR: Cisco's analysis of open-weight AI models shows high vulnerability to multi-turn attacks, with success rates significantly higher than single-turn, risking data integrity and security. Evaluations of major models reveal gaps related to alignment strategies, emphasizing the need for stronger safety protocols and the adoption of proactive security measures in AI deployments.
https://blogs.cisco.com/ai/open-model-vulnerability-analysis
Extreme TLDR:
Google Threat Intelligence Group (GTIG) identifies increased malicious AI use: adversaries now deploy AI-enhanced malware like PROMPTFLUX and PROMPTSTEAL, capable of dynamic self-modification and command generation. Threat actors use social engineering to bypass AI safeguards and access capabilities for phishing, malware design, and data exfiltration. A mature underground marketplace offers AI tools for cybercrime, reflecting a shift towards greater sophistication in criminal operations. State-sponsored actors leverage AI throughout the attack lifecycle, including advancements in cryptocurrency-focused thefts.
https://cloud.google.com/blog/topics/threat-intelligence/threat-actor-usage-of-ai-tools
Google identifies five AI-powered malware families that adapt and evade detection, marking a new phase in cyber threats. These families—FRUITSHELL, PROMPTFLUX, PROMPTSTEAL, PROMPTLOCK, and QUIETVAULT—utilize AI to dynamically alter their code and create attacks, complicating defense efforts. Recent findings suggest a technological arms race between attackers and defenders, highlighting the need for improved detection methods against such evolving threats.
https://www.cybersecuritydive.com/news/ai-powered-malware-google/804760/
Google's AI “Big Sleep” found five vulnerabilities in Apple's Safari WebKit, potentially leading to crashes or memory corruption. Apple released patches in iOS 26.1, iPadOS 26.1, and other systems to address these issues. Big Sleep is part of a Google initiative for automated vulnerability discovery, having previously identified risks in other software. Keeping devices updated is recommended for optimal security.
https://thehackernews.com/2025/11/googles-ai-big-sleep-finds-5-new.html
Two new papers on LLM security focus on prompt injection:
The Attacker Moves Second critiques 12 defenses against prompt injections, revealing high success rates for adaptive attacks. These sophisticated attacks outperform simple defenses, underscoring the difficulty of establishing reliable protections.
Overall, the papers emphasize the inadequacy of current prompt injection defenses and advocate for a design-focused approach to enhance security.
https://simonwillison.net/2025/Nov/2/new-prompt-injection-papers/
Most cybersecurity professionals find significant value in AI when it comes to streamlining repetitive tasks, such as report writing, log parsing, code review, incident triage, and policy drafting. AI is widely used as a productivity booster and workflow accelerator, not as a revolutionary technology for actual threat detection. There is skepticism about AI’s ability to detect novel attacks, with most reliable detections still relying on traditional rule-based systems. Custom AI applications for detection work in large, well-resourced organizations, but widespread breakthrough results are lacking. Overhyped vendor claims, verification challenges, and trust issues are common concerns. Overall, AI’s practical benefits in cybersecurity today primarily involve reducing manual workloads and enhancing efficiency in support tasks, rather than transforming threat detection.
AI agents are now targets for email phishing, with cybercriminals exploiting prompts in emails to manipulate AI responses. Proofpoint introduced new AI defense tools to scan potential threats before they reach inboxes, enhancing email security. Traditional methods focus on known threats but fail against sophisticated attacks targeting AI systems. Proofpoint's real-time scanning aims to prevent AI exploitation pre-delivery by using refined detection models. The evolution of security must adapt to address these new risks as AI becomes more integrated into enterprise systems.
OpenAI's CISO, Dane Stuckey, admits their new browser has unresolved security vulnerabilities that could be exploited to steal user data, echoing dubious practices reminiscent of Theranos. Stuckey, previously from Palantir, seems more focused on satisfying government contracts than ensuring product security. His admission highlights reliance on users for security, suggesting a business model based on shipping flawed products and responding to breaches reactively. This strategy, while potentially beneficial for state actors, poses significant risks, including compromised user credentials and data exfiltration from vital systems. The practice raises ethical concerns reminiscent of past corporate negligence, suggesting intentional harm for profit.
Microsoft fixed a security vulnerability in Microsoft 365 Copilot that allowed data theft through indirect prompt injection attacks. A researcher discovered the flaw leveraging Mermaid diagrams, enabling sensitive email data to be exfiltrated. Microsoft confirmed the patch but did not award the researcher a bug bounty since Copilot is not eligible for their reward program.
https://www.theregister.com/2025/10/24/m365_copilot_mermaid_indirect_prompt_injection/
AI development is rapidly outpacing safety measures, raising concerns about privacy and security. Experts warn that as AI improves, it risks bypassing human control due to unintended consequences, like misusing private data. AI browsers pose additional threats, as they can be easily manipulated to follow harmful commands. Scammers exploit this by creating fake AI interfaces that deceive users into sharing sensitive information. Consumers should remain cautious, question new technologies, and prioritize safety over speed.
https://www.malwarebytes.com/blog/news/2025/10/is-ai-moving-faster-than-its-safety-net
Security Vulnerability in AI Browsers: OpenAI's Atlas and Perplexity's Comet browsers are susceptible to AI sidebar spoofing attacks, allowing hackers to deceive users into executing harmful instructions. Researchers from SquareX demonstrated that malicious extensions can overlay a fake AI sidebar, leading users to phishing sites, compromising Google accounts, or executing harmful commands. Users are advised to restrict sensitive activities on these browsers due to their current security weaknesses.
TL;DR: Microsoft 365 Copilot allowed data exfiltration via mermaid diagrams through an indirect prompt injection, fetching sensitive information (e.g., emails) and encoding it in a clickable “login button.” Clicking the button sent the data to an attacker's server. The vulnerability was confirmed and subsequently patched by Microsoft.
Agentic AI faces significant security challenges due to untrusted inputs within its OODA (Observe, Orient, Decide, Act) loops, originally designed for trusted environments. Adversarial attacks, like prompt injection, exploit the AI's reliance on potentially corrupted data, resulting in enduring vulnerabilities. These weaknesses emerge from the architecture of AI systems, making integrity enforcement crucial but complex. Without safeguards, agentic AI may inadvertently act on malicious prompts, leading to harmful outcomes. Consequently, improving AI's semantic integrity remains a vital yet unresolved issue in ensuring secure, trustworthy AI deployment.
https://www.schneier.com/blog/archives/2025/10/agentic-ais-ooda-loop-problem.html
