Description du poste
Basically:
• You could probably make a superintelligent AI with some currently unknown new paradigm for <$10k if you had the right hardware and algorithms• Plausibly this would be much easier and faster with better hardware• LLMs will make developers of novel-paradigm AI systems really good at hardware creation and design• This seems bad, because those novel-paradigm AIs seem unlikely to be aligned with human valuesASI could in principle be created cheaplyI think it's very plausible that with the right hardware and algorithms someone could make a wildly superintelligent AI model extremely cheaply (<$10k in energy cost), for roughly the reasons described in this post (although I think it underestimates how useful hardware improvements will be for building ASI). Basically, I think that given that you can train a human from nothing to a 20-year-old for less than $10k worth of electricity, you should expect to be able to train an AI to a similar level for the same amount of electricity with the right hardware and algorithms. In fact you should be able to do that much more efficiently, since you only care about intelligence and don't need to worry about powering a whole human body. And if you're making your own brain it's probably pretty doable to just make it bigger.
I don't think we'll get to this kind of novel ASI through iteratively improving current AI paradigms, but instead using something that's very different than current LLMs. To be clear, I also think that scaling LLMs could get us to superintelligence. I'm worried that ASI will be created through some novel paradigm first though.
Probably misalignedThis seems pretty bad!
It's hard to speculate about the alignment properties of an AI created through a training process that's unknown to me! However, it seems unlikely to me that if an ASI is created through novel, extremely cheap methods it would be aligned with human values. With today's LLM scaling, AI labs spend a lot of time trying to make their models aligne