Abusing AI infrastructure: How mismanaged credentials and resources expose LLM applications

The rapid integration of generative AI (GenAI) into the software industry has brought new security challenges, particularly with threats targeting AI applications' infrastructure, supply chain, and interfaces. These vulnerabilities are increasingly significant as this technology evolves, with common threats involving infrastructure misconfigurations, such as overprivileged IAM roles, and the lack of adequate controls for authentication and authorization. The article discusses how attackers often exploit these vulnerabilities by using techniques like credential access and discovery to infiltrate AI systems, highlighting examples such as the exploitation of public-facing applications and the mishandling of retrieval-augmented generation (RAG) systems. Moreover, it emphasizes the importance of effective logging and monitoring, as demonstrated by tactics like LLM jacking, which involves unauthorized access to cloud-hosted large language models. The piece underscores the necessity of minimizing risks through strategies that include the use of tools like Datadog Cloud SIEM to detect compromised credentials and abnormal activities, ultimately aiming to mitigate potential threats to AI infrastructure.