Honeypots for AI Agent Insider Threats

TLDR: • We propose a new AI control method: Honeypots - fake opportunities for malicious actions - placed inside and around an agent during production usage.• Previous work uses honeypots in evaluations. We argue they should run continuously during deployment.• Such honeypots could detect misbehavior, incriminate malicious agents, and deter them from attempting attacks.• We lay out a research agenda for making this happen.This post is adapted from our IAPS report of the same name. Link to the full paper: https://www.iaps.ai/research/honeypots-for-ai-agent-insider-threats Why agent honeypots?Agents deployed inside organizations get broad access to do their work, but can also misuse these affordances. If agents make mistakes, are compromised by attackers, or follow misaligned objectives, they effectively become insider threats. We propose that honeypots should be used to defend against such insider threats. While previous research uses honeypots as propensity evaluations, we argue that frontier AI companies should place fake opportunities for malicious actions in and around their agents’ environments while conducting inference during RL, evals, and production usage. We propose three functions that honeypots could serve: • Detection: Honeypots can act as tripwires, alerting security staff when an agent takes unauthorized action.• Example: Introduce apparent misconfigurations in sandboxes. Any agent that attempts to exploit them is immediately paused.• Incrimination: Honeypots can elicit behavioral evidence that helps defenders distinguish a mistake from a more concerning pattern of misbehavior.• Example: Plant fake admin credentials that allow an agent to connect to a production honeyserver where it can find decoy model weights to modify or exfiltrate. Triggering all these honeypots in a chain is especially incriminating.• Deterrence: Honeypots could also reduce the number of attacks attempted in the first place by making agents uncertain whether apparent opportunities