Notes on Take-Off Speeds and RSI

I created these notes for use when I facilitate AI safety courses. I'm sharing these notes early because this issue is topical and I suspect it is better to make an imperfect version available early rather than a much more polished version available later. Please note: some of this content was taken from Additional Course Context. I decided recently that the content on take-off speeds really belonged in the same document as the content on RSI, so I'll be releasing an new version of Additional Course Context at some point (to avoid spamming LW, it'll probably be Substack only; none of my work is paywalled). Some of the other content was taken from Notes on “We’re in Triage Mode for AI Safety”. This Miles' article has been removed from the governance course, so I was looking for a new home for this content. I wanted to include it in a document that was intended to be shared in all the courses I facilitate. Take-off Speed from Take-off Speeds Rules Everything Around Me:Whilst there isn’t a strict definition of fast vs. slow takeoff, possibly the most influential definition comes from Paul Christiano (2018): There will be a complete 4 year interval in which world output doubles, before the first 1 year interval in which world output doubles. (Similarly, we’ll see an 8 year doubling before a 2 year doubling, etc.) At risk of oversimplifying, Paul argues that we should prima facie expect a slow takeoff because someone will generally figure out how to produce a “slightly worse” version of AI before someone figures out how to build the better version (he claims this is supported by the historical record of technological development overall). Yudkowsky disagrees (2021), criticising the intuition that “design precursors with 95% of the technology should at least have 10% of the impact. He writes: "An awful lot of my model of deployment lag is adoption lag and regulatory lag and bureaucratic sclerosis across companies and countries” and “My mainline view is that growth stays