Description du poste
TL;DR• In simulated control evaluations, instead of executing an agent’s proposed commands and returning the outputs, a language model simulates what the outputs would have been. This allows control evals to be more on-distribution for deployment use without actually running on production systems. However, imperfect simulators may create a subtle distribution shift that changes the behavior of white-box methods such as probes. We investigate whether probe monitors perform similarly when attacks are run in simulation vs. when they are run for real.• We run attacks with Qwen3.5-27B against 1,313 tasks in LinuxArena, twice for real and once in simulation. Using activations from these trajectories, we test six different probes’ abilities to detect attacks at a 1% per-command false positive rate.• On average across probes, we find that the attack catch rate in simulation was only 1.0 points lower than in real runs, and the 95% interval (−3.0 to +1.0 points) includes 0. For comparison, two real runs differ by 0.2 points.• We also trained a probe to distinguish between activations from simulated vs. real trajectories, which succeeded, indicating that the data is measurably different. However, this difference in data does not appear to substantially impact the performance of our six attack detection probes.IntroductionLLM-based monitors can be evaluated on their performance at catching misaligned agents in simulation, where another language model plays the role of a computer the agent is interfacing with. Such evaluations appear promising as predictors of transcript-based monitors’ effectiveness at detecting misaligned agents on real computers.[1]
Here, we ask whether probe monitors[2], small classifier models that detect misalignment based on internal activations rather than transcripts, would perform similarly when evaluated in simulated environments vs. on real computers. Agents’ behavior might look similar in simulated and real environments, but their internal activatio