← Toutes les offres
AV

Human organisations as an intuition pump for AI's unit of agency

Alexandre Variengien

Global
Publié le 21 novembre 2026

Description du poste

Tl;dr: Intuitions we have about human organisations, like how we think of making deals or how we think about the power or values of an organisation, can be productively applied to understand groups of AIs taking coordinated action in the world. An important consideration in understanding the type of AI organisations we expect is which notion of self AIs will adopt. I conclude by applying these notions to consider how many AI units of agency we should expect in the future. What's an AI?There is no single definition for what should be considered an AI. The boundary of what counts as an AI varies widely depending on what we are talking about: it's often the model weights ("Fable 5.2 is very smart"), or a specific instance in a scaffolding ("my Codex just finished the app"), or a persona shared by many models ("Claude hedges too often"). I argue that these concepts are too anchored in the computational substrate and not very helpful for understanding the unit of agency of AI systems made of multiple instances acting in the world. None of the three options above can easily answer questions such as: What does it mean for an AI to have power? What are the values of an AI? If we want to make a deal with an AI, who makes the deal? When an AI takes over, how do we think about the entity taking over? If AI enters into conflicts, how do we think about the parties? I propose to think about AI entities acting in the world as analogous to human organisations. We have many rich examples of humans acting collectively around a shared purpose, from freelancers to nation-states or social movements. I believe we can reuse many of these intuitions to think about AI systems. I’ll focus on groups of AI instances taking coordinated action in the world, which I’ll call AI organisations (AI orgs). When discussing AI takeover threat models (e.g. AI 2027, Cotra 2022 …), it is often assumed: “one set of model weights = one big agent = one AI organisation”. While there are reasons to believe this