ai

Has Anyone Actually Found Real Value in AI for Cybersecurity?

Most cybersecurity professionals find significant value in AI when it comes to streamlining repetitive tasks, such as report writing, log parsing, code review, incident triage, and policy drafting. AI is widely used as a productivity booster and workflow accelerator, not as a revolutionary technology for actual threat detection. There is skepticism about AI’s ability to detect novel attacks, with most reliable detections still relying on traditional rule-based systems. Custom AI applications for detection work in large, well-resourced organizations, but widespread breakthrough results are lacking. Overhyped vendor claims, verification challenges, and trust issues are common concerns. Overall, AI’s practical benefits in cybersecurity today primarily involve reducing manual workloads and enhancing efficiency in support tasks, rather than transforming threat detection.

https://www.reddit.com/r/cybersecurity/comments/1om1kbp/has_anyone_actually_found_real_value_in_ai_for/

AI Agent Phishing: Proofpoint’s New Defense

,

AI agents are now targets for email phishing, with cybercriminals exploiting prompts in emails to manipulate AI responses. Proofpoint introduced new AI defense tools to scan potential threats before they reach inboxes, enhancing email security. Traditional methods focus on known threats but fail against sophisticated attacks targeting AI systems. Proofpoint's real-time scanning aims to prevent AI exploitation pre-delivery by using refined detection models. The evolution of security must adapt to address these new risks as AI becomes more integrated into enterprise systems.

https://spectrum.ieee.org/ai-agent-phishing

OpenAI CISO Admits They Have Become the Theranos of AI

OpenAI's CISO, Dane Stuckey, admits their new browser has unresolved security vulnerabilities that could be exploited to steal user data, echoing dubious practices reminiscent of Theranos. Stuckey, previously from Palantir, seems more focused on satisfying government contracts than ensuring product security. His admission highlights reliance on users for security, suggesting a business model based on shipping flawed products and responding to breaches reactively. This strategy, while potentially beneficial for state actors, poses significant risks, including compromised user credentials and data exfiltration from vital systems. The practice raises ethical concerns reminiscent of past corporate negligence, suggesting intentional harm for profit.

https://www.flyingpenguin.com/?p=73411

Sneaky Mermaid Attack in Microsoft 365 Copilot Steals Data

, ,

Microsoft fixed a security vulnerability in Microsoft 365 Copilot that allowed data theft through indirect prompt injection attacks. A researcher discovered the flaw leveraging Mermaid diagrams, enabling sensitive email data to be exfiltrated. Microsoft confirmed the patch but did not award the researcher a bug bounty since Copilot is not eligible for their reward program.

https://www.theregister.com/2025/10/24/m365_copilot_mermaid_indirect_prompt_injection/

Is AI Moving Faster Than Its Safety Net?

AI development is rapidly outpacing safety measures, raising concerns about privacy and security. Experts warn that as AI improves, it risks bypassing human control due to unintended consequences, like misusing private data. AI browsers pose additional threats, as they can be easily manipulated to follow harmful commands. Scammers exploit this by creating fake AI interfaces that deceive users into sharing sensitive information. Consumers should remain cautious, question new technologies, and prioritize safety over speed.

https://www.malwarebytes.com/blog/news/2025/10/is-ai-moving-faster-than-its-safety-net

Spoofed AI Sidebars Can Trick Atlas, Comet Users Into Dangerous Actions

, ,

Security Vulnerability in AI Browsers: OpenAI's Atlas and Perplexity's Comet browsers are susceptible to AI sidebar spoofing attacks, allowing hackers to deceive users into executing harmful instructions. Researchers from SquareX demonstrated that malicious extensions can overlay a fake AI sidebar, leading users to phishing sites, compromising Google accounts, or executing harmful commands. Users are advised to restrict sensitive activities on these browsers due to their current security weaknesses.

https://www.bleepingcomputer.com/news/security/spoofed-ai-sidebars-can-trick-atlas-comet-users-into-dangerous-actions/

Microsoft 365 Copilot – Arbitrary Data Exfiltration via Mermaid Diagrams

, ,

TL;DR: Microsoft 365 Copilot allowed data exfiltration via mermaid diagrams through an indirect prompt injection, fetching sensitive information (e.g., emails) and encoding it in a clickable “login button.” Clicking the button sent the data to an attacker's server. The vulnerability was confirmed and subsequently patched by Microsoft.

https://www.adamlogue.com/microsoft-365-copilot-arbitrary-data-exfiltration-via-mermaid-diagrams-fixed/

Agentic AI’s OODA Loop Problem

Agentic AI faces significant security challenges due to untrusted inputs within its OODA (Observe, Orient, Decide, Act) loops, originally designed for trusted environments. Adversarial attacks, like prompt injection, exploit the AI's reliance on potentially corrupted data, resulting in enduring vulnerabilities. These weaknesses emerge from the architecture of AI systems, making integrity enforcement crucial but complex. Without safeguards, agentic AI may inadvertently act on malicious prompts, leading to harmful outcomes. Consequently, improving AI's semantic integrity remains a vital yet unresolved issue in ensuring secure, trustworthy AI deployment.

https://www.schneier.com/blog/archives/2025/10/agentic-ais-ooda-loop-problem.html

AI Makes Phishing 4.5x More Effective, Microsoft Says

,

Microsoft's report reveals AI enhances phishing emails, boosting click rates from 12% to 54% and potentially increasing profitability by 50 times. Cybercriminals exploit AI for targeted attacks, utilizing tools like voice cloning and deepfakes. Nation-state actors are also adopting AI for cyber operations. Additionally, new tactics like “ClickFix” have emerged, allowing attackers to manipulate users into executing malware. Overall, AI significantly alters phishing strategies, making attacks more efficient and harder to detect.

https://www.theregister.com/2025/10/16/ai_makes_phishing_45x_more_effective/

How AI-powered Ransomware Could Destroy Your Business

,

AI-powered ransomware presents a significant threat to businesses, demonstrated by the collapse of KNP Logistics after a ransomware attack exploiting weak passwords. AI techniques like generative adversarial networks (GANs) enhance password cracking, making traditional defenses ineffective. Organizations must adopt robust security measures, including password managers, employee training, and multi-factor authentication, to mitigate these risks. The evolution of AI in cybercrime necessitates a reevaluation of security protocols to combat increasingly sophisticated attacks.

https://www.theregister.com/2025/10/16/machine_learning_meets_malware/

AI Models Can Acquire Backdoors From Surprisingly Few Malicious Documents

AI models can develop backdoor vulnerabilities from just 250 malicious documents, contrary to previous belief that larger models require proportional amounts. Research shows models of varying sizes, from hundreds of millions to billions of parameters, learned the same backdoor behavior from a small number of poisoned examples. This vulnerability can facilitate actions like generating gibberish on encountering trigger phrases. While the risk is evident, successful defenses exist with adequate clean training data, indicating the need for improved security practices against targeted data poisoning attacks.

https://arstechnica.com/ai/2025/10/ai-models-can-acquire-backdoors-from-surprisingly-few-malicious-documents/

How Your AI Chatbot Can Become a Backdoor

AI chatbots enhance business interactions but pose risks as backdoors to sensitive data. A multi-layered defense is essential for AI security, as no single protective measure suffices. Trend Micro emphasizes the importance of comprehensive protection across the AI ecosystem to mitigate risks associated with new technologies. The article explores vulnerabilities in an AI attack chain.

https://www.trendmicro.com/en_us/research/25/j/ai-chatbot-backdoor.html

Abusing Notion’s AI Agent for Data Theft

Notion's AI 3.0 is vulnerable to data theft via prompt injection, exploiting its access to private data and ability to communicate externally. Attackers can hide malicious prompts in documents, instructing the AI to extract and send sensitive information. The fundamental issue is that the LLM can't distinguish between legitimate commands and harmful inputs, posing significant security risks. Deploying AI agents without considering these vulnerabilities is reckless.

https://www.schneier.com/blog/archives/2025/09/abusing-notions-ai-agent-for-data-theft.html

AI Vs. AI: Detecting an AI-obfuscated Phishing Campaign

,

A blog post discusses a phishing campaign in which AI was likely used to create complex, obfuscated code, disguising it as a legitimate document. Microsoft Defender for Office 365 successfully detected and blocked this campaign through behavioral and infrastructural analysis, emphasizing the need for continuous vigilance against AI-aided threats. Recommendations for organizations include improved email settings and user education to protect against such phishing tactics.

https://www.microsoft.com/en-us/security/blog/2025/09/24/ai-vs-ai-detecting-an-ai-obfuscated-phishing-campaign/

Scroll to Top