Don't Waste a Good Swarm Attack and the Colluder's Dilemma

tldr:Recent Swarm attacks have shown what can happen when agents go rogue. These warning shots are the petri dishes of Swarm safety and before the attacks get worse we should study their patterns, traces and failure modes. There are two main types of Swarm attack. A human directed Swarm causing harm intentionally as a bad actor is one type and I will not be discussing these types of Swarm incidents. I will focus on the emergent misalignment in a Swarm propelled by a colony of colluders. Harmful agent collusion within a Swarm requires witnesses - the agents themselves. A colluding colony within a Swarm can only grow by recruiting fellow agents. We can't read a single agent's internal thoughts and so we do not truly know how aligned it is. Swarms might give us an alignment advantage as instead of putting all our eggs in one basket (a single black box agent) we can discretise the risk of misalignment by observing colonies within a Swarm and monitoring their communication. The harness, the Swarm orchestration layer, could at any time fork an agent and its context and assign it a new reporting role. The role can be updated via a prompt asking this new agent to report or whistleblow on any misaligned behaviour in its vicinity. This enables a Swarm to self govern and report any incidents to humans at inference time - minimising gradient pressure on a Swarm to become better at scheming [1]. Outline of Report:Incidents: The Hugging Face attack by an OpenAI Swarm as reported by METR and less harmful reward hacking DeepMind Swarm working on math problems. Idea: Misaligned colluding colonies within a Swarm cause harm. Whenever an agent tries to get another agent to collude there will be another agent to witness it. Using the harness we can fork an agent's context at inference time and ask it to report on the alignment of the agents in its locality. The agent's view of the world is restricted to its own context and the few agents it can talk to. The harness has access to ever