Description du poste
Interpretability is considered by many a major bottleneck for alignment. As a result, a lot of people who are concerned about AI safety, and especially about extinction risk and other catastrophic risks, work on interpretability.
In this post, I want to argue that better interpretability methods would likely speed up capabilities dramatically. Worse, they would speed up capabilities much more than they would speed up alignment. This is true for the current world, and for any world without extremely high levels of coordination and sanity about ASI development.
Under any worldview where faster capabilities progress makes catastrophe more likely, we should expect progress in interpretability to make the odds of catastrophe much worse.
To put it briefly, this is because the better we get at interpretability, the more AI will be able to read and optimize its own code. This means that improvements in interpretability would catalyze recursive self-improvement (RSI).
If you are concerned about AI x-risk, and you work on interpretability because of it, I urge you to think about this and consider stopping.
Chess as a toy example of an AI capability
As a field, "interpretability" is a loose collection of related research agendas rather than something well defined, so I want to pin down what I mean by it.
In this post, interpretability is anything that helps you understand the internals of models and how they think. I'm going to use chess as a toy example of a task performed by an AI. The point of this exercise is to show how methods that help to understand how the AI is playing chess would allow one to massively improve AI's chess ability, given the same amount of compute and training data.
[1]
Specifically, let's think about how a modern general-purpose AI thinks about chess. Just so you know, they are quite decent at it: Opus 5.5 on high thinking plays at roughly 2100 Elo (the level of a strong amateur). At this setting, Opus 5.5 thinks for single-digit min