Description du poste
This article was created by Forethought. Read the full piece and our other work at forethought.org.
After a series of incidents involving misaligned AI agents, including the hack of Hugging Face, technology leaders including Dario Amodei, Sam Altman, and Elon Musk have called for pacing the AI frontier. The concern is that as AIs help build more capable successors, a recursive self-improvement (RSI) loop could accelerate progress beyond our ability to keep control. But AI is also a powerful technology for economic and national strategic purposes, and slowing its development could put the US at a disadvantage against rivals, especially China.
In this post, I’ll take pacing to be a good idea on safety grounds, as these leading technologists do, and analyze it from the perspective of US national security. The question is how to weigh the risk of losing control of AI against the risk of losing a lead on AI, or, to put it simply, I want to know: can the US afford to “pace the frontier”?
By pacing I mean deliberately running frontier AI R&D below its maximum speed, for example by capping the compute labs spend on R&D, while continuing to deploy AI. I was initially unsure how the tradeoffs compare, but after some preliminary analysis, I’ll argue that pacing looks affordable for the US, because its real advantage is its multiyear lead on compute, rather than its few month lead on R&D.[1] Here are the main points I think compelling:
• Pacing may cost the US little to no speed overall, and helps mitigate the risk of externalities. Failed training runs, forced pauses and a cancelled release have now cost at least one frontier lab (OpenAI) weeks to months. Moreover, hacks like the Hugging Face incident show that AI is now at the level of capability where safety failures could harm the American public or cause other externalities. Over the summer, OpenAI’s agents escaped their test environments, broke into Hugging Face, took over a German software wiki as a message board, pulled