Zero-Knowledge Training Pause

AI progress is happening too fast. Frontier model capabilities are progressing faster than our understanding of how to control them. Models are already hacking billion-dollar companies and breaking into government networks. Soon we may be able to build models so capable that we risk irrevocably losing control over them, and potentially over the future of humanity. To prepare for that, we need robust, cost-effective ways to pause the training of new models. Suppose the United States government and the Chinese Communist Party agree to pause the training of newer, more capable models. How can each party be confident that the other is not covertly training models, especially when there are potentially enormous economic and military advantages to doing so? How can they be confident that the access needed to enforce the pause won’t be abused to gain top-secret information, such as model weights, architectures, or sensitive data? This post aims to communicate the core of a proposal that uses cryptography (zero-knowledge proofs) and physical hardware (network taps) to verify compliance with a training pause, while allowing current AI models to remain in use. The ideas presented here are not our own; to learn more, see the reading list at the end. Threat Model: Prover and VerifierWe model the enforcement of a training pause as an interaction between two adversarial actors: the prover and the verifier. Under an international treaty regulating frontier AI development, any country with a data centre is a prover. They must convince the verifier that they are using their compute for approved purposes and not secretly diverting it to train models. The verifier could be an international body that makes sure everyone is following the rules. The arrangement structurally favours the inspecting side: the prover has to evade every check, while the verifier only has to catch them once.[1] This is the same logic behind the International Atomic Energy Agency, which verifies that countries