Description du poste
Try the app here: whoseside.web.app. You can chat with a model and try to work out which who it secretly favours.
With the recent Hugging Face attack, we're relying more and more on LLMs to audit the trails and safeguard the actions of other LLMs. That makes it vital to be sure our LLMs really are the helpful assistant they're presented ( as they can easily be sleeper agents.[1] ). We need a way to keep our agents working for us. Or at least a way to tell if it goes wrong.
Compared with buggy code, buggy AI is a lot harder to define or solve.[2] Mechanistic interpretability has some promising leads[3], but is still nowhere close to being able to point out novel misalignment. If we look at the field of cybersecurity, first a vulnerability comes out or an attacker discovers something new, and then defenses get built. I find that's a good analogy for machine learning as well, where we've seen this pattern with adversarial attacks on images and jailbreaking prompts. Currently, open-weight models offer no guarantee that they are helpful-only or aligned, besides our ability to evaluate them.[4] You're essentially trusting that nowhere in the model training supply chain has a vulnerability, bug, or secret loyalty been inserted or post-trained into the weights. Current sleeper-agent / secret-loyalty research says as much.[5]
[6]
To demonstrate this problem, I built a little website where you can try to audit and find the secret loyalties instilled into these models. For those familiar with the prompt-injection game Gandalf, this is a similar idea. Try it here: whoseside.web.app.[7]
The code is available at [repo link TODO]. This work was funded by a BlueDot Impact Rapid Grant.
• ^Sleeper Agents (Hubinger et al., Jan 2024): backdoored behaviour survives standard safety training · Auditing language models for hidden objectives (Anthropic, Mar 2025): a model trained with a hidden objective, found by 3 of 4 blind audit teams · Simple probes can catch sleeper agents (Anthropic,