Description du poste
OverviewIn my previous posts, I discussed and attempted to define AI loyalty as a distinct trait of AI agents and explored its relationship to other existing fields of AI safety. In this post, I aim to expand on that discussion and outline a proposed research road map toward AI alignment through the development of loyal AIs.
At the outset, it is important be recognize that research into AI loyalty is inherently dual use and therefore must be pursued responsibly. With that in mind, I propose dividing this research agenda into a primary and a secondary line of investigation. The primary research direction would focus on investigating whether loyalty in AI agents is effective in counteracting loss of control risks. This would require work on developing methods to measure and strengthen AI loyalty, as well as, experiments to evaluate whether increasing the loyalty of AI agents reduces loss of control risks. The secondary research direction would experiment with strategies and governance structures that would reduce the concentration of power risks inherent in building loyal AIs.
Primary TrackThe central premise of this research agenda hinges on two hypotheses:
1. Loyalty is easier to build into AI agents than general alignment.
2. AI loyalty reduces the risk of loss of control.
If either of these hypotheses are false, then this research agenda should be abandoned in favor of more promising research directions
For this research agenda, we will first seek to show that loyalty can be used to reduce loss of control risks. This will involve, building evals to assess model loyalty, building environments to assess loss of control, and building techniques for increasing the loyalty of models.
Evaluating LoyaltyThe first step to making progress on this agenda is to put together evals that fully encapsulate our definition of loyalty in AIs and which will serve as targets for training. In our previous post we distinguished between different loyalty classes to distinguish betwee