Description du poste
I used to think of "automating alignment" as a natural category. But when I tried to think through the strategic implications, I realised that it depends a lot on the type of work. I'm now thinking of it in three buckets:
• Empirical alignment research: design and execution of alignment-relevant experiments (for a given notion of what we’re aiming for)• Conceptual alignment research: theoretical or judgement-driven work to improve our understanding of what we’re aiming for and how to achieve it (including how to measure it)• Direct alignment work: making particular AI systems more aligned (including via inputs such as training data)In the rest of this post I'll walk through my current takes on these in terms what’s needed for automation, how good or bad they look, what our priorities might be (to the extent we're interested in automating alignment[1]), etc.
Automating empirical alignment research
This is the most obvious fit for the AI capabilities we’re getting at the moment. You want to know more about the relationship between some alignment-relevant variables? Great, an AI agent can design and run that experiment for you. You want to know what training methods help your alignment benchmark go up? In the not-too-distant future, AI agents will probably be able to do this too. When we have the automated capabilities researchers we seem on track for, this is exactly the kind of thing they’ll likely be able to do a lot of.
Note that I’m not counting the design of benchmarks/metrics/evals as part of this bucket. In current practice that’s tightly intertwined with empirical alignment research, and often the hardest part of it. But for automation purposes it clusters much more with conceptual work; it’s easy to imagine automating the rest of empirical research without it (as AI capabilities generally get automated); and I think its automation has different strategic upshots.
The capabilities that are needed for automating empirical alignment research very heavily overlap