← Toutes les offres
F

Three types of automated alignment work

Forethought

Global
Publié le 16 novembre 2026

Description du poste

If we’re thinking about automating alignment, it may help to distinguish different types of work we may wish to automate: • Empirical alignment research: design and execution of alignment-relevant experiments (for a given notion of what we’re aiming for)• Conceptual alignment research: theoretical or judgement-driven work to improve our understanding of what we’re aiming for and how to achieve it (including how to measure it)• Direct alignment work: making particular AI systems more aligned (including via inputs such as training data)These buckets are actually pretty different, in terms of what’s needed for automation, what the impacts of automating are, how much damage subtle misalignment within the automated systems could do, etc. So I’ve come to think that we should be tracking them separately. Automating empirical alignment researchThis is the most obvious fit for the AI capabilities we’re getting at the moment. You want to know more about the relationship between some alignment-relevant variables? Great, an AI agent can design and run that experiment for you. You want to know what training methods help your alignment benchmark go up? In the not-too-distant future, AI agents will probably be able to do this too. When we have the automated capabilities researchers we seem on track for, this is exactly the kind of thing they’ll likely be able to do a lot of. Note that I’m not counting the design of benchmarks/metrics/evals as part of this bucket. In current practice that’s tightly intertwined with empirical alignment research, and often the hardest part of it. But for automation purposes it clusters much more with conceptual work; it’s easy to imagine automating the rest of empirical research without it (as AI capabilities generally get automated); and I think its automation has different strategic upshots. The capabilities that are needed for automating empirical alignment research very heavily overlap with the capabilities needed for automating AI capabilities r