Description du poste
Subtitle: This mad race must end.
Note: I think this is partially me working through my plans and partially my emotions, so imo it's less epistemically rigorous than ideal for LessWrong.
I might have less than a year left to affect the insane, reckless, Faustian race to ASI, and I plan to spend that time doing everything I can to push for a pause.
I currently think there’s a decently high chance that by default, there’s 6-12 months left before my personal point of no return: the point after which my own actions no longer plausibly affect the future. This is different from other people’s points of no return: for example, people like Dario Amodei (CEO of Anthropic) or Donald Trump may have slightly longer times to wake up and change course. This is also different from other timelines that people sometimes throw around, like timelines to AGI or ASI or fully automated remote-worker or whatever. However, it is the most important for my own decisions.
I do not defend this position here. Other people have written partial explanations that I like (Karthik, AI Futures Project, Sentinel, Rutger Bregman). I may also write more accessible explainers in the near future.
Regardless, humanity is absolutely not ready for what comes next, whether that’s human extinction, complete loss of control to rogue AIs or a few oligarchs, or something far weirder.
While I don’t have strong confidence that I actually only have 6-12 months left, I think very few people are working on this by default, and AI Safety is growing very quickly (though still far smaller than needed), and newcomers to the field can cover the scenarios where we have more time.
Given this, I think my best strategy is to push hard for a pause on training AI: specifically a complete moratorium on training stronger AI models than the ones we currently have and severe limitations on AI R&D, until scientists, policymakers, and the general public have more time to understand what’s going on and catch up.
I think pushing for a