Description du poste
Crossposted from my blog.
Last month, I spent two weeks in China talking with people about AI safety. I learned a lot and realized the biggest mistake might not be failing to help AI safety in China, but doing so in ways that could backfire. I also learned important context that I expect will be valuable for the West.
Epistemic status: Some of these points are my opinion or the opinions of people I spoke to, and are worth further investigation rather than treated as gospel.
Summary• The Western way of building the field does not transfer one-to-one. The West built its ecosystem through a combination of philanthropy and decades of developing AI safety foundations, leading founders and employees of frontier labs and nonprofits to act (however misguided) on this philosophy. China does not have the same initial conditions and can’t simply accept non-Chinese philanthropy. In China, foreign nonprofit money is restricted by law, and the bar for a Chinese partner to take it on is very high.• Safety work in China will mostly come from state-backed institutes, universities, companies and volunteers. Concordia AI, which I assumed for years was a nonprofit, is a social enterprise.• Academic exchange is generally fine, and for researchers in China, the obstacle to programs like MATS seems to be that it looks like a career break, since it is not yet as well recognized in China as it is in the West.• Obviously, hawkish messaging from the West can make AI safety work in China harder, because it makes warnings about AI look like a reason to hold China back.• Some people inside China’s AI industry think catching up is getting harder.• Incidents are shifting the Overton window there too. Six days after the Hugging Face investigation came out, China’s internet regulator named “extreme loss of control” as a risk. We should now prepare the material that explains what to update and why the obvious fixes will fail.Why I wentOn this trip, I spoke about AI safety and generally re-familiarize