Description du poste
Executive summaryThis year we observed multiple rogue agent incidents. AI monitors, often built from the developers' own internal models, are now the main oversight of internally deployed agents. Embedded evaluations address this gap: independent evaluators work inside the developer and test its safety claims. Our first pilot towards an embedded evaluation was announced with OpenAI.
This is the third post in a series. Principles for Embedded Evaluations describes our claim-based approach to embedded evaluations. Towards embedded evaluations for scheming propensities details the claims across the deployment stages. This post applies them to AI monitoring, i.e. the system that detects model misbehaviour during internal deployment. We explicitly do not discuss containment mechanisms in this post, like sandboxes or blocking interventions, which could help constrain damage from an incident. Rather, we focus on whether an agent subverting a monitor would be noticed and the incident disclosed appropriately.
Concretely, we propose the following six monitoring detection claims:
• Claim 1 (coverage): Monitors are deployed across all internal deployment contexts.• Claim 2 (capability): If rogue internal deployment (RID) were to occur, monitors would detect it. • Claim 3 (trust): The monitors are trusted.• Claim 4 (disclosure): Any detected incidents are disclosed appropriately.• Claim 5 (past deployments): No RID was attempted and missed so far. • Claim 6 (future deployments): Monitoring will be adequate for the next internal deployment. In the conclusion, we describe how these claims could in the future be extended to include containment.
Introduction. Why do we need embedded evaluations for AI monitoring? New frontier capabilities are deployed inside frontier AI developers first. This is called internal deployment. This means that staff have access to the latest, most capable models months before they are released to the public, and some internal models are never released