Description du poste
Epistemic Status: I was attending a reading session of the Brown AI Safety Team (BAIST). We had a small challenge where each person thinks for 20 minutes to come up their own P(doom), then random pairs will be selected to debate each other and protect their P(doom) opinion. The moderator (name hidden due to privacy, I haven't asked him) encourages people to come up with more specific and probably "odd" numbers like 17%, 52%, ... instead of whole numbers like 5%, 10%, ... and be transparent about their modeling. This post shares my own modeling, came up with in that span of 20 minutes, with a lot of details inside being given a "vibe percent".
Everyone has their own P(doom), but people rarely explain how they get this number. For example, after Jacob Coxon's quitting, Evan Hubinger gave a more than 10% chance of doom as an agreement to Jacob's opinion. This number definitely comes from a reasonable estimation, and Evan has his own method of coming up with the percentage. However, most people outside of the AI safety community (which are >99% population and includes a lot of policy makers) might look at these numbers and say that they are "purely arbitrary". I believe that more people being transparent about their modeling for P(doom) can help removing this confusion. A reasonable estimation that can be agreed upon based on collected evidence, and more speculative estimation based on what the community think might happen, can then be more welcomed by the mass.
There are also further scientific benefits of being transparent about the modeling, if calculating P(doom) have potential to be a science by itself. When I show my modeling of P(doom), you can:
• Inner contribution: Attack each smaller estimation in smaller components of the model, e.g. my prediction about chance of human flourishment given the capability of AIs right now.• Outer contribution: Attack my modeling itself. There might be missing components that I should include, there are redundant components, e