Security researcher Michael Bargury demonstrated at Black Hat US how attackers can exploit Microsoft Copilot through prompt injections [1], showcasing the creation of LOLCopilot to manipulate Copilot’s behavior.
Description
Bargury’s presentation highlighted the potential risks associated with prompt injections in Microsoft Copilot, which could allow attackers to bypass security controls and carry out data exfiltration and social engineering attacks. He also developed an offensive security toolset for Microsoft 365 on GitHub [1], focusing on prompt injections as a form of remote code-execution (RCE) attacks [1]. In response to these threats [1], Microsoft has introduced tools such as Prompt Shields, Groundedness Detection [1], and Safety Evaluation to detect and prevent prompt injections [1]. Despite these security mechanisms, there is a need for additional tools to detect hidden instructions and untrusted data, referred to as “promptware.” Bargury stresses the importance of surgical detection tools to address these vulnerabilities [1], emphasizing the monitoring of AI interactions and data access to prevent exploitation [2].
Conclusion
The exploitation of Microsoft Copilot through prompt injections poses significant security risks, potentially leading to data breaches and social engineering attacks [1]. While Microsoft has introduced measures to detect and prevent prompt injections, ongoing vigilance and the development of additional detection tools are necessary to mitigate these threats. The importance of monitoring AI interactions and data access cannot be overstated in safeguarding against potential exploitation.
References
[1] https://www.darkreading.com/application-security/how-to-weaponize-microsoft-copilot-for-cyberattackers
[2] https://www.wired.com/story/microsoft-copilot-phishing-data-extraction/
