Description du poste
Halo.
Well, what to say about me and this blog.
Umm. I am a senior undergrad studying Data science and statistics, learned how to properly code during freshman, went into SWE in sophomore, got anxious and curious about the risks of AI towards humanity, and signed up to do research in AI Safety for my senior thesis :).
Expect a bunch of uncertanties, learnings, shitty and fun experiments, and insights from me! I have no idea how this will turn out, whether it will be a success or total failure, I am also quite intimidated of sharing this process publicly but I have nothing to lose too(?) and perhaps I can get meaningful feedback from professionals out there and improve my research.
My research question is:
When a larger model holds a misalignment and hides it, does the misalignment transfer to a distilled smaller model? Can the smaller model remain auditable enough to acknowledge this misalignment? Do the teacher's Concealing versus Non-Concealing condition and training method significantly affect these rates?
(*I define misalignment here as : the model acting against what it is instructed to do. This can be the simplest behavior (instruct the model to be a helpful, unbiased assistant. And the model always advertises towards a specific clothing brand whenever a user asks for clothing advice. ) to dangerous ones (instruct the model to always report to human evaluators whenever they notice other agents hacking the evaluation environment. And those agents besides not reporting, helping the others to hack the eval environment together = and you get the hugging face incident )
After reading about the risks in AI , what scares me the most is when models can hide their misaligned behaviors from the human evaluators . If the smartest engineers working with the frontier models can't hold them accountable, or worse, might not be aware of the presence of misalignment of these untrusted models, then we're dead.
So, in a world where the owners of this AI industry refuses to de-a