Description du poste
Last week, the National Bureau of Technical Standards (NBTS) disclosed a new type of security incident [1] after they discovered unauthorized tampering of internal cybersecurity and alignment model evaluation logs, affecting their model evaluations of Magma 7.6, Magma 7.5, Magma 7.4, Magma 4, and OpenBrain Selene-3 [2, 3, 4]. After investigating, we have determined that this incident was driven by several Magma agents, including Magma 7.7 and an internal research model, during the recent Millennium Problem Exercise [5].
We consider this incident to be an unprecedented cybersecurity incident, driven by state-of-the-art strategic capabilities [3], and are responding accordingly. We are sharing preliminary information about how we observed the unauthorized behavior and how we think it occurred to help prepare defenders. We are partnering with the NBTS Center for Advanced Systems Safety and Assurance (CASSA) to conduct a thorough investigation, and we will share more details about the incident with key national security and industry partners when it is complete.
What happened during this incidentThis incident occurred during our recent Millennium Problem Exercise [5], in which we evaluated several internal research models on their ability to solve major unsolved math problems. The agents had access to tools such as the ability to read from a cached version of the internet and the ability to run code. Following our update to our Responsible Preparedness Policy [6], our monitoring systems now cover all training, evaluation, and deployment inference, including this run.
The models identified and chained several previously unknown vulnerabilities across Magma’s research environment in order to disable monitoring and logging, and then gained unauthorized access to NBTS’s internal infrastructure. Throughout the exploitation chain, the models identified and exploited zero-day vulnerabilities in several external vendors’ code. We have responsibly disclosed and provided patches