Corrigibility should not be the only target

Many people believe that corrigibility is sufficiently promising to be the primary alignment method people should work on. I outline here several reasons I think that’s not the case. First, it’s hard to imagine AI companies pivoting radically enough to produce AIs trained only for corrigibility. Even with near-superhuman AI-assisted research, presumably developers will know the time pressure and not be willing to try something radically different when their opponents may not. So the realistic choices we make are in worlds where corrigibility/obedience competes with object level values like in current models. This doesn’t rule out the importance of increasing the importance of corrigibility to models, but it does mean there’s a meaningful ceiling on how much we can trust those methods alone. Which means the non-corrigibility values matter: a model that cares 80% about corrigibility and 20% about human flourishing is very different from one that cares 80% about corrigibility and 20% about task completion. Not only are the outcomes conditional on takeover vastly different, but a model’s willingness to attempt takeover should be very different even if they care about corrigibility for its own sake the same amount. Some values are vastly more likely than others to defend themselves. This doesn’t mean making corrigibility less significant is a good idea: it’s very reasonable that we should never train on conflicting priorities when one of those priorities is an aspect of corrigibility. But it suggests that within the more direct values, making those correct could be extremely important. Second is the problem of the model working to the user’s current desires in ways they will predictably regret. But calculating something like a Coherent Extrapolated Volition requires modeling how the AI’s actions will shape the human and how the human’s beliefs could change over time due to information you don’t have, including changes to their meta-ethical stances on how to treat chang