Description du poste
An AI model that appears aligned could be either (1) actually aligned or (2) hiding its misalignment while being evaluated. How can we tell which?
One idea: Look at the trajectory of eval results across the entire training process.
My hypothesis is that misaligned behavior would emerge before the ability to hide misalignment, like how young children first misbehave in obvious ways before learning to be sneaky. As training progresses, we'd first see misalignment eval scores rise, then see them drop as the model learns to hide misalignment. Even if the model becomes very sophisticated - say, modifying its chain of thought or internal activations to fool white-box evals - we should see the misalignment earlier in training. This seems likely, though it is debatable and worth testing.
This analysis of the training trajectory is one tool among many. Of course we want to see black-box and white-box alignment evals on the final model, using whatever tools we can to understand the model's true motivations. But a model might still develop sophisticated hiding strategies that defeat even white-box evals. Inspecting the training trajectory is a backstop against this.
This point was made previously by Apollo Research in this post. (Their chart even matches mine almost exactly, though I promise I first made mine independently.)
In this post, I'll describe a simple project exercising this strategy. With the model and eval that I chose, we don't see the up-then-down trend in measured misalignment that would indicate hiding. This is unsurprising since the model is well below frontier scale and the eval likely doesn't capture misalignment that well. The sample sizes are small and statistical uncertainties are high. But this approach is promising with improvements such as different evals, larger models, and deliberately misaligned test models.
This is my project for the BlueDot Technical AI Safety Project. My main purpose has been to get started with some hands-on AI inf