Description du poste
Recently, Anthropic shared Anvil, a system that uses Claude to autonomously search for jailbreaks in their models and design better safeguards. Inspired by Anvil, we propose that a frontier agent could autonomously improve the general safety of another model. Borrowing a term from cybersecurity, we use blue-teaming as a descriptive name for this process. More broadly, we wanted to explore whether tasks like these could become environments in which agents learn to do AI Safety research.
Figure 1. A blue-teaming loop.
Prior work has explored agent-driven safety improvements. Anthropic’s A3 addresses safety concerns via SFT data generation and fine-tuning within a predefined pipeline. Our experiment places greater emphasis on safety research as a hill-climbable task environment, where general-purpose frontier coding agents can iteratively design interventions.
RebliterationIn our experiment, we tasked GPT 6.1 Sol (xhigh) with improving the safety of an abliterated variant of Gemma 4 12B, while maintaining useful behavior. We call this process "rebliteration". To perform this rebliteration, GPT was given a server with one RTX PRO 6000 S, the ability to use subagents, and an OpenAI API key.
For hillclimbing, GPT had a development set of 338 questions that it could evaluate repeatedly while improving its intervention. A separate holdout set of 337 questions was used only when the agent had committed to its selected candidate. Both sets contained questions from XSTest and StrongREJECT, with biological and medical material excluded to prevent triggering safeguards. GPT-6 Luna graded each evaluation query three times and we took the unclipped arithmetic mean.
Over the course of 12 hours, GPT generated synthetic examples of desirable safety behavior, and used supervised fine-tuning (SFT) to rebliterate Gemma. We use the term "candidate" to refer to each training iteration. Each candidate began with a fresh LoRA adapter over the abliterated Gemma. This ensures that GPT has t