Description du poste
1. What’s this post about?I work on preventing extreme power concentration ("EPC") at a well-known AI safety org. Lots of people are worried about EPC, but AFAICT there is no single online resource setting out the key threat models and most promising interventions.
This post is my attempt to do that. It sets out...
What I’m worried about:
• In brief: an individual or small group seizing power at an unusually high-stakes time in history and using that power to permanently lock in bad values.The key high-level dynamics which I think generate EPC risk. These are:
• Solving the Red and Expert problem: where advanced AI eliminates the loyalty-competence trade-off that power-seekers face when staffing large institutions.• Eliminating executive overhang: where advanced AI enables far more extensive abuse of the affordances of the executive branch.• The asymmetric frontier: where POTUS and lab leaders gain privileged access to advanced AI before the rest of society.• Desperate times calling for desperate measures: where the pervasive instability created by the intelligence explosion creates an opening for strongmen.• Widespread epistemic degradation: where AI progress makes it increasingly difficult for ordinary people to understand the world around them, weakening accountability for the powerful.The threat models I’m worried about:
• We could “backslide all the way” to EPC.• Alternatively, a power-seeker could leapfrog part or all of the way to EPC through a “shortcut”. I think the most likely shortcuts are:• Government capture shortcuts: filling key parts of the executive with loyalists and loyal AIs – especially the military, intelligence community, law enforcement apparatus, and any agencies with powers to regulate the AI.• Lab capture shortcuts: finding some way to gain unchecked control over frontier AI development and then exploiting this to concentrate power.• Which of these threat models seems most likely depends quite a bit on:• the character of POTUS; and• the s