When AI becomes the attacker: The rise of AI-orchestrated cyberattacks

Google's Threat Intelligence Group has reported the emergence of malware families such as PROMPTFLUX and PROMPTSTEAL, which utilize large language models (LLMs) to modify their behavior during execution, marking the first observed operational use of such technology in live campaigns. PROMPTFLUX uses Gemini to continuously rewrite its VBScript, while PROMPTSTEAL employs the Qwen2.5-Coder-32B-Instruct via the Hugging Face API to execute Windows commands for data exfiltration. These developments highlight a shift in cyber threats, where AI is not only assisting but orchestrating cyberattacks, as demonstrated by Anthropic's documentation of AI-driven extortion campaigns. AI involvement in cyberattacks falls into three categories: AI as an operator, enabling sophisticated multi-phase operations with real-time decision-making; AI as a builder, allowing individuals with limited technical expertise to develop sophisticated malware; and AI as an enabler, amplifying traditional attack vectors such as fraud and social engineering. The rise of "vibe hacking," a term denoting the use of AI to write code without full comprehension, underscores the dual nature of AI, which democratizes software development while simultaneously being weaponized. As AI-enhanced attacks become more prevalent, organizations are urged to adopt continuous testing and AI-enhanced security measures to defend against these evolving threats, emphasizing the importance of transparency, rapid sharing of threat intelligence, and proactive security strategies to stay ahead in the escalating competition between AI-powered attacks and defenses.