Quantity has a quality of its own

People love to argue about whether AIs could recursively self-improve their way to superintelligence, each improvement making the next one easier as progress accelerates. I'm unsure. It could be that progress stalls at near-human or somewhat superhuman levels, that the ultimate qualitative limits of intelligence are not particularly beyond the human-level, or that the quantity of AI researchers involved is so vast that substantially superhuman AIs are created without strictly entering a super-exponential regime. But honestly, I think recursive self-improvement is a little overrated. Even if human-level AIs do not rapidly lead to superhuman AIs, the vast quantities of them could be very dangerous. Arguments against recursive self-improvement provide little consolation here. Conversely, recursive self-improvement to superintelligence does seem particularly dangerous, and pausing or pacing it may be desirable. But stopping it may not be enough, because vast numbers of less-than-superintelligent agents could still be destabilizing and catastrophic. Soon after we build human-level AIs, we will be able to run hundreds of millions of them, and before long billions. At that scale, quantity has a quality of its own. There will soon be a lot of human-level AI systems Human-level AIs—by which I mean AIs able to competently perform most white-collar work—appear plausible within the next few years, and when they arrive I expect they'll be cheap to run. On the typical task today's AIs can already do, they cost a few cents per hour of human work. Most jobs are sequences of such tasks, so a human-level AI worker should need well under one high-end chip's worth of compute. About 31 million of these chips had been sold by June 2026, enough for hundreds of millions to several billion workers. Alternatively, we can count how many of today's agents our chips could run. Epoch estimates that chips shipped through 2027 could run 30 to 170 million frontier agents at once, or 140 to 720