Reviewing my 2017 AI Risk Project

In 2017, after getting laid off from a startup, I spent a while looking into AI risk: Many people I respect have started to think that AI risk is the most valuable place to focus EA efforts. For example, 80000 Hours ranks it first on their "list of global issues", taking into account "scale, neglectedness, and solvability". On the other hand, I have a lot of friends working in machine learning, and none of them think AI risk is worth working on now. This level of disagreement is very strange, and kind of worrying. After playing for a dance weekend in South Carolina where casual conversation among non-tech people included earnest speculation over the likelihood that AI gets us all killed, I thought it might be a good time to look back at when this was really speculative. Summary: I talked to the right people, heard some incredibly prescient ideas, and got the bottom line wrong. I do a lot of my thinking via serial writing, and in this case that was a series of twelve blog posts. Taking them one at a time: Looking into AI Risk (2017-06-26). This held up well. I picked something well worth looking into, identified a lot of relevant resources, and got a lot more good reading recommendations in the comments. Superintelligence Risk Project (2017-07-03). The main part of this post were five ideas about where the disagreement might be. These worked ok as jumping off points for interviews, but they weren't as core to disagreements as I hoped: people often ended up with similar views on most of these ideas while still being far apart in their views on how we should prioritize AI risk. One thing I do like, though, is that these were pretty concrete questions which now do have answers: How likely is it that current approaches are all we need for AGI, with relatively straightforward extensions and a lot of scaling? Very likely. When I asked this question, "current approaches" meant deep neural nets and reinforcement learning. The past nine years have mostly b