Description du poste
Thank you to @Thane Ruthenis for discussing this topic with me previously. Some arguments in this essay are taken from that discussion, but all mistakes and/or bad arguments should be attributed to me. You can view the original discussion here (optional for reading this essay). Thank you as well to @grainsilo for feedback on an earlier version.
Consider the following argument for discussing AI capabilities as a safety-focused person:
• Understanding AI capabilities better has many strategic benefits, particularly for timelines.• Yes, good ideas tend to break containment, so some of our AI capabilities analysis may get read by capability researchers…but at the same time, these people have already thought about these questions more deeply and thoroughly than we have.• Therefore, none of our arguments are new to capability researchers.• Therefore, we will have no impact on advancing AI capabilities or timelines by discussing this topic.I’ve seen this argument made explicitly in some places, I presume it's made in many people’s heads whenever they post about capabilities, and I’ve certainly made this argument to myself before.
I now also think this argument is wrong. I don’t really know what to do about this conclusion. Maybe the answer is nothing, but I don't know that, so it seems like a good idea to make people more aware of the reasons why I think this, in the hopes that it prompts further discussion, reflection, and caution.
Problem 1: High-Surface-Area QuestionsCertain fields/problems/questions can be aptly described as being “high-surface-area”: complex, with many interlocking pieces, lots of potential connections to other fields/problems/questions, layers of better abstractions/concepts/ontologies waiting to be peeled back, and multiple plausibly helpful (although not necessarily ultimately helpful) lines of thinking that can be pursued with respect to the question.
Some examples of high-surface-area questions:
• The alignment problem• Please Don’t Throw Your