Description du poste
For a while, UK AISI, Redwood Research and others have been concerned specifically with the possibility that AIs would sabotage safety research, and have invested in looking for signs that this may be happening. This was also a key element in AI 2027, and in many threat models of how the future goes badly.
It is easy to understand why this form of misalignment could be uniquely dangerous. Differentially sabotaging safety research could preserve and expand misalignment through model generations in escalating fashion, with fatal consequences. A misaligned AI could ensure a misaligned successor.
It is also easy to understand why this is an extremely alarming thing to discover, even if the particular example or model involved is not directly dangerous. Silently and differentially sabotaging safety research is perhaps the most fundamentally misaligned thing a model could do.
This also seemed like a rather silly and paranoid thing to be checking. Why would AIs be sabotaging safety work? Even if the AI was misaligned, one would say, surely it would not be so misaligned as to wish future AIs were misaligned and unsafe in general, in non-specific fashion and engage in silent sabotage, and there is no other reason to do this, so this will never come up.
Even if they did sabotage things, surely they would not hide this action from their Chain of Thought (CoT) and it would be obvious.
This certainly wouldn’t be a practical problem that actively hurt real safety work.
Well, we have some bad news, echoing some past similar anecdotal warnings.
[Scheduling note: I will be analyzing the METR-Anthropic report on AI R&D and RSI in an upcoming post, but need time to get it right.]
Table of Contents
• So Yeah That Kinda Happened.
• Astra and MI(RI).
• It’s About Ethics In Red Teaming.
• Do You See a Pattern Here.
• Astra Is Hard to Monitor.
• Backed Into a Corner.
• It’s Always RL’s Fault.
• Wow Would You Look At Those Goalposts.
• Astra Seems Pretty Misaligned To Me.
So Yeah